Tool Use in Language Models: From ReAct to Native Tool Calling

1. Definition and thesis

A language model “uses a tool” when it emits an instruction to an external system, receives the system’s output, and incorporates that output into later generation. The tool may be a calculator, a search endpoint, a retrieval index, a Python interpreter, a browser, a SQL database, a robotics controller, a payment API, or another model. The important distinction is not whether the tool is complex, but whether the model’s next state depends on an observation produced outside its own parameters.

In early systems, tool use was mostly a prompting convention. A model produced text like Action: Search[query]; an external controller parsed that text, executed a call, and appended an Observation: into the context. In modern native APIs, the model emits structured tool-call objects, often validated against JSON Schema or provider-specific schemas, and the application returns structured tool results through reserved message fields or content blocks. OpenAI’s current function-calling documentation describes this as connecting models to external systems and data through function tools and hosted tools, with tool calls, tool outputs, and call identifiers forming the execution lifecycle. OpenAI Platform Anthropic similarly defines tool use as a contract in which Claude emits a structured request, while either the client application or Anthropic’s infrastructure actually runs the operation and returns the result. Claude Platform Google’s Gemini documentation likewise frames function calling as a way to connect models to external tools and APIs, allowing a model to decide when to call functions and provide parameters for them. Google AI for Developers

The thesis of this article is that tool use is best understood as environmental grounding for model reasoning, not as a synonym for agency. Tool use gives a model access to fresh information, exact computation, persistent state, and side-effectful operations. But Agentic Systems require additional machinery: goals, control loops, memory, permissions, delegation, monitoring, rollback, and evaluation. Native tool calling made tool use easier to deploy, but it did not solve planning, judgment, safety, or long-horizon reliability.

2. The conceptual arc: from text-only inference to model-environment loops

The pre-tool language model was largely closed over its parameters and prompt. It could answer from training data, infer from the local context window, and simulate procedures in text. That made it powerful for linguistic and commonsense tasks, but brittle for arithmetic, current facts, private data, exact symbolic manipulation, and operations that require real-world state. Tool use emerged because many failures of pure language modeling are not failures of “intelligence” in the abstract; they are failures of interface.

A calculator is better than a transformer at exact arithmetic. A search engine is better than parametric memory at fresh facts. A database query is better than a prose answer when the user needs data from a private warehouse. A code interpreter is better than hand-simulated execution for nontrivial programs. A file search system is better than asking a model to “remember” a document it has never seen. The core insight behind Tool-Augmented Language Models is therefore modular: let the language model plan, translate user intent, and synthesize results, while specialized systems execute operations for which they are more reliable.

A minimal tool-use loop has four phases:

Phase	Model-side behavior	System-side behavior	Typical failure
Selection	Decide whether a tool is needed and which one to call	Present available tools and constraints	Unnecessary call, wrong tool, missing call
Argument construction	Fill typed arguments or free-form command text	Validate syntax, schema, permissions	Malformed JSON, wrong type, hallucinated argument
Execution and observation	Pause generation while the tool runs	Execute tool and return result	Tool error, stale result, unsafe side effect
Integration	Use the result in subsequent reasoning or response	Optionally trace, evaluate, or continue loop	Ignored result, over-trusting result, loop repetition

This loop is simple, but it changes the epistemology of model output. The answer is no longer only “what the model says.” It becomes a composed artifact: model prior, prompt, tool selection, tool semantics, execution state, result formatting, and final synthesis. That composition is the source of both the power and the fragility of tool use.

3. Precursors: MRKL, ReAct, PAL, Toolformer, Gorilla

3.1 MRKL: route language tasks to expert modules

MRKL Systems — “Modular Reasoning, Knowledge and Language” — articulated an early architecture for combining language models with external neural and symbolic modules. Karpas et al. described MRKL as a system approach that places a language model inside a flexible architecture containing multiple neural models and discrete reasoning or knowledge modules. arXiv

MRKL’s enduring contribution is the router view. A general model does not need to solve every subproblem internally. It can dispatch factual lookup, arithmetic, symbolic reasoning, or domain-specific operations to modules. This anticipates native tool calling, but MRKL was more of an architectural proposal than a standardized API pattern. It also foregrounded a tension still present today: the more modules a system exposes, the harder tool selection becomes.

3.2 ReAct: interleave reasoning traces with actions and observations

ReAct made tool use feel like a reasoning process rather than a static pipeline. Yao et al. proposed prompting language models to generate reasoning traces and task-specific actions in an interleaved sequence. In their framing, reasoning traces help the model induce, track, and update action plans, while actions allow the model to interface with external sources or environments such as Wikipedia or interactive tasks. arXiv

The canonical ReAct loop looks like this:

Thought: I need to know X.Action: Search[X]Observation: ...Thought: The result suggests Y, but I need Z.Action: Lookup[Z]Observation: ...Answer: ...

This was not native function calling. It was text that an external harness interpreted. But the pattern was profound because it joined Chain-of-Thought Prompting with environment interaction. In HotpotQA and FEVER, ReAct used a Wikipedia API to reduce hallucination and error propagation relative to reasoning-only prompting; in ALFWorld and WebShop, the paper reported absolute improvements of 34% and 10% over imitation- and reinforcement-learning baselines with only one or two in-context examples. arXiv

ReAct also exposed a design problem that modern APIs still inherit: visible reasoning traces are not the same as reliable control. The model may rationalize a bad action, get stuck in loops, misread an observation, or call a tool when ordinary answer synthesis would be cheaper. ReAct gave the field a powerful interface pattern, not a guarantee of agency.

3.3 PAL: make the program the intermediate reasoning object

Program-Aided Language Models took a different route. Instead of using a model to simulate arithmetic or algorithmic reasoning in prose, Gao et al.’s PAL generated programs as intermediate reasoning steps and offloaded execution to a Python runtime. The paper argued that language models are often good at decomposing a problem but bad at performing the final logical or arithmetic operations correctly. arXiv

PAL is tool use in a narrow but important sense: the tool is an interpreter. The model writes code; the runtime computes. Across mathematical, symbolic, and algorithmic tasks, the PAL paper reported that Codex with PAL surpassed much larger chain-of-thought models, including a 15 percentage-point absolute top-1 improvement over PaLM-540B with chain-of-thought on GSM8K. arXiv

PAL helped establish a recurring empirical lesson: when the task has an exact executable substrate, do not ask the model to pretend to be that substrate. Ask it to produce a representation that the substrate can execute.

3.4 Toolformer: train models to decide when and how to call APIs

Toolformer moved from prompting to self-supervised tool-use learning. Schick et al. argued that language models struggle with basic functionality such as arithmetic and factual lookup, where smaller specialized tools can perform well, and introduced a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to incorporate results. The APIs included a calculator, question-answering system, two search engines, translation system, and calendar; the training procedure used only a handful of demonstrations to bootstrap self-supervised annotation. arXiv

Toolformer’s key idea was not merely “call tools.” It was learn the affordance boundary: detect when an external operation will improve continuation likelihood enough to justify insertion. This matters because tool use has costs. Every call consumes context, latency, and sometimes money or risk. A useful tool-using model must know not only how to call tools, but when not to.

3.5 Gorilla: API calling as a knowledge and retrieval problem

Gorilla focused on API calls rather than calculators or search snippets. Patil et al. argued that API tool use remains hard even for strong models because it requires accurate input arguments and avoidance of hallucinated APIs. Gorilla fine-tuned a LLaMA-based model to write API calls and combined it with retrieval over documentation, reporting that the retriever helped adapt to test-time documentation changes and mitigate hallucinated API usage. arXiv

Gorilla matters because real tool use is rarely just “call calculator.” Modern production systems expose many functions with overlapping names, optional parameters, hidden preconditions, version drift, and ambiguous user intent. In that world, tool use becomes a documentation retrieval problem, a schema-following problem, and a semantic mapping problem.

3.6 API-Bank, ToolBench, and BFCL: from demos to benchmarked tool use

By 2023–2026, evaluation shifted from toy tool calls to broad tool-use benchmarks. API-Bank introduced a benchmark with 73 API tools, 314 tool-use dialogues, 753 API calls, and a training corpus built from 2,138 APIs across 1,000 domains; its authors reported that GPT-4 excelled in planning but still left room for improvement. ACL Anthology ToolLLM and ToolBench scaled the surface further, collecting 16,464 REST APIs across 49 categories from RapidAPI and training ToolLLaMA with a retriever to use them. arXiv

The Berkeley Function Calling Leaderboard, especially BFCL V4, evaluates function-calling ability across real-world data, serial and parallel calls, abstention, stateful multi-step interactions, and periodically updated tasks. Its paper notes that stateful multi-turn tasks, memory, dynamic decision-making, and long-horizon reasoning remain open challenges even as leading models do well on single-turn calls. Proceedings of Machine Learning Research

The trajectory is clear: the field moved from “can a model call a search API?” to “can a model reliably choose, parameterize, sequence, abstain from, and recover around many tools under changing conditions?”

4. Comparison of precursor systems

System	Date	Core contribution	Tool abstraction	Empirical signal	Limitation
MRKL	2022	Modular architecture combining LMs with neural and symbolic expert modules	Router to expert modules	Conceptual architecture for hybrid systems	Tool routing and module selection remain hard arXiv
ReAct	2022	Interleaved reasoning, action, observation loop	Prompt-level actions parsed by a harness	Improved QA faithfulness and interactive-task performance	Text protocol is brittle; reasoning traces do not guarantee control arXiv
PAL	2022	Generate executable programs instead of doing symbolic work in prose	Python runtime as execution tool	Strong gains on math, symbolic, and algorithmic tasks	Works best when task admits executable formalization arXiv
Toolformer	2023	Self-supervised learning of API-call insertion and use	Inline API calls inserted into text	Improved zero-shot performance while preserving language modeling	Limited tool set; not a full production protocol arXiv
Gorilla	2023	API-call generation grounded in retrieved documentation	APIs as callable functions with docs	Surpassed GPT-4 on API-call generation in reported setting	API correctness depends on retrieval, docs, and evaluation scope arXiv
BFCL	2024–2026	Standardized, periodically updated function-calling evaluation	Native and prompted function-calling tasks	Tracks accuracy, cost, latency, abstention, multi-step state	Long-horizon and stateful tasks remain hard Proceedings of Machine Learning Research

5. Native tool-calling APIs

Native tool calling changed the implementation layer. Instead of asking the model to emit a textual pseudo-command, developers describe tools to the model through API fields, the model emits structured tool-call objects, the application executes calls, and results are returned in a reserved format. This gives developers better validation, logging, routing, schema constraints, and security boundaries.

5.1 OpenAI: function calling, hosted tools, and structured outputs

OpenAI introduced function calling for gpt-4-0613 and gpt-3.5-turbo-0613 in June 2023. Developers could describe functions to the model, and the model could choose to output a JSON object containing arguments conforming to the function signature. OpenAI’s announcement described use cases such as calling external tools, converting natural language into API calls or database queries, and extracting structured data. OpenAI

The current OpenAI tooling surface distinguishes several related concepts. Function calling lets the model call developer-defined functions; built-in or hosted tools include capabilities such as web search, file search, code execution, remote MCP, image generation, and tool search. OpenAI’s docs define a lifecycle around tool calls and tool outputs, including call identifiers that associate a tool result with a prior call. OpenAI Platform

OpenAI also separates Structured Outputs from function calling. Structured Outputs can constrain final responses to a schema, while function calling bridges the model to application functions, databases, or user-interface actions. JSON mode ensures valid JSON but does not by itself enforce schema adherence; Structured Outputs are intended to enforce a supplied JSON Schema more directly. OpenAI Developers In OpenAI’s strict mode for function tools, setting strict: true makes tool calls adhere to the function schema and requires constraints such as additionalProperties: false and all fields being required. OpenAI Platform

The direction is toward a unified tool platform rather than a single “function calling” feature. OpenAI’s Agents SDK describes agents as applications that plan, call tools, collaborate across specialists, and maintain state, while the SDK runtime manages turns, tools, guardrails, handoffs, sessions, and related orchestration. OpenAI Developers

5.2 Anthropic: tool_use, tool_result, client tools, server tools

Anthropic’s Claude API uses explicit content blocks such as tool_use and tool_result. Claude decides whether to call a tool based on the user request and tool descriptions, returns a structured call, and then continues after the client or server provides the result. Anthropic distinguishes client tools, which the application executes, from server tools, which Anthropic runs on its infrastructure. Claude Platform

This design makes the model’s non-execution role explicit: Claude does not run arbitrary code just because it emits a tool call. It asks for a tool to be run. The application or provider executes, applies permissions and validation, and returns the observation. Anthropic’s docs also document error conditions when a tool_use block is not followed by corresponding tool_result blocks, emphasizing that the message protocol itself is part of correctness. Claude Platform

Anthropic’s engineering materials are especially explicit about tool-use ergonomics. Their “writing effective tools for agents” post frames tools as a contract between deterministic systems and nondeterministic agents, noting that an agent may choose the wrong tool, pass wrong parameters, use too many tools, or process responses incorrectly. It recommends fewer, clearer tools, tight schemas, response truncation or pagination, and evaluation metrics that include runtime, number of tool calls, token consumption, and tool errors. Anthropic

Anthropic has also pushed tool discovery and execution toward larger tool ecosystems. Its 2025 advanced tool-use materials describe Tool Search, Programmatic Tool Calling, and Tool Use Examples to reduce context bloat, improve parameter selection, and handle many MCP servers or large tool surfaces. The post reports that loading several MCP servers can consume tens of thousands of tokens, and that tool search reduced context from roughly 72,000 to 8,700 tokens while preserving most of the necessary tool information in an internal evaluation. Anthropic

5.3 Google Gemini: function calling, built-in tools, modes, and context circulation

Google’s Gemini tooling documentation frames tools as extensions that let models take action, access real-time information, perform computation, and use specialized capabilities. Built-in tools include Google Search, Maps, Code Execution, URL Context, Computer Use, and File Search; Google’s docs explicitly describe File Search as enabling retrieval-augmented generation. Google AI for Developers

Gemini function calling follows the familiar pattern: the developer declares functions, the model returns a structured function call and parameters, the application executes the function, and the function response is sent back so the model can answer or call another tool. Google’s Vertex AI documentation exposes tool-calling modes such as AUTO, VALIDATED, ANY, and NONE, allowing developers to control whether the model may call functions, must call functions, or must avoid them. Google AI for Developers

Google’s newer Interactions API suggests an architectural shift toward persistent, observable, multi-step interaction state. Its documentation describes server-side history, complex multi-turn workflows, execution steps containing model thoughts, server and client tool calls, tool results, and final output. Google AI for Developers Google also describes “context circulation” for combining built-in and custom tools in Gemini 3, where outputs from one tool can be circulated into subsequent tool calls or final responses. Google AI for Developers

5.4 Vendor comparison

Provider	Main protocol shape	Execution location	Schema/control features	Built-in tools	Agent direction
OpenAI	Tool calls and tool outputs, including function tools and hosted tools	Client for custom functions; provider for hosted tools	JSON Schema, strict mode, Structured Outputs, tool choice, tool search	Web search, file search, code execution, remote MCP, image generation, tool search	Agents SDK manages turns, tools, guardrails, handoffs, sessions OpenAI Platform
Anthropic	tool_use and tool_result content blocks	Client tools or Anthropic server tools	Input schemas, strict mode, tool-use examples, tool search, programmatic calling	Web search, code execution, memory/computer-style tools, server tools depending on product surface	Emphasis on tools as contracts and agent-oriented tool design Claude Platform
Google Gemini	Function declarations, function calls, function responses	Client for custom functions; provider for built-in tools	Modes such as AUTO, VALIDATED, ANY, NONE; context circulation	Google Search, Maps, Code Execution, URL Context, Computer Use, File Search	Interactions API with server-side state and observable execution steps Google AI for Developers+2Google Cloud Documentation+2

The common pattern is now stable: describe capabilities, constrain calls, execute outside the model, return observations, continue generation. The differences are in message format, schema strictness, hosted-tool portfolio, state management, and how aggressively each provider folds tool use into agent runtimes.

6. When tool use beats pure reasoning

Tool use is not universally better than pure generation. It helps when the task has an external substrate that is more reliable, more current, more authorized, or more computationally exact than the model’s parametric memory. It hurts when tool calls add unnecessary latency, token cost, complexity, or failure surfaces.

6.1 Exact computation and symbolic manipulation

Tool use is strongest when correctness depends on exact execution. PAL is the cleanest evidence: when math or symbolic tasks can be represented as Python programs, generating code and executing it often beats prose reasoning. The PAL result on GSM8K — a 15-point top-1 improvement over a much larger chain-of-thought baseline — is an early signal that external execution can dominate scale for certain task types. arXiv

The same principle generalizes to code interpreters, SQL queries, spreadsheets, theorem provers, and simulation engines. The model is useful as an intent translator and decomposition engine; the tool is useful as an executor. The failure boundary appears when the model writes the wrong program, encodes the wrong assumptions, or treats execution output as more relevant than it is.

6.2 Factual lookup, current information, and private knowledge

Tool use also helps when the answer is outside the model’s reliable memory. ReAct’s use of a Wikipedia API reduced hallucination and error propagation in HotpotQA and FEVER by grounding intermediate steps in retrieved observations. arXiv Retrieval-Augmented Generation formalized a related pattern: combine parametric memory with non-parametric memory, such as a dense Wikipedia vector index accessed through a retriever, to produce more factual and specific outputs than a parametric-only baseline. arXiv

Modern RAG, web search, and file search are therefore not separate from tool use. They are read-only tool use: the action is a query, the environment is a corpus or search index, and the observation is a set of passages, snippets, documents, or citations. The special feature of RAG is that the tool usually has no side effect and is often placed early in the pipeline, before final generation.

6.3 API execution and workflow automation

Tool use becomes essential when the user wants an action rather than an answer: book an appointment, create a ticket, send an email draft, update a CRM record, run a build, deploy code, query a database, or operate a browser. This is where native tool calling matters most, because side-effectful operations require typed arguments, permissions, audit logs, user confirmation, and recoverable error handling.

Gorilla, API-Bank, ToolBench, and BFCL all show that API use is a distinct skill from ordinary answer generation. Gorilla emphasizes correct API names and arguments; API-Bank evaluates planning through tool-use dialogues; ToolBench scales to thousands of APIs; BFCL evaluates native function calling across serial, parallel, abstention, and stateful tasks. Proceedings of Machine Learning Research+3arXiv+3ACL Anthology+3

6.4 Interactive environments

ReAct’s results on ALFWorld and WebShop point to another tool-use regime: the tool is not merely a function but an environment. The model observes state, acts, observes again, and eventually succeeds or fails. This resembles reinforcement learning or classical planning more than question answering. arXiv

The key distinction is that one-shot tool calling is a local operation, while environment interaction is trajectory-level behavior. A model may be competent at a single function call but poor at a 30-step task requiring memory, recovery, exploration, and changing plans. BFCL’s note that stateful, long-horizon, and dynamic decision-making tasks remain open challenges is consistent with this distinction. Proceedings of Machine Learning Research

6.5 When tools add overhead

Tool use adds overhead in at least six ways: extra prompt tokens to describe tools, extra output tokens for calls, latency for external execution, error-handling complexity, attack surface, and opportunity cost when the model could have answered directly. Anthropic’s docs explicitly note that tool use introduces token usage from tool definitions, tool calls, tool results, and provider-specific system prompts. Claude Platform Anthropic’s advanced tool-use materials also show how large tool surfaces can consume tens of thousands of tokens before the model has done any real task work. Anthropic

The empirical pattern is therefore conditional:

Task type	Tool use likely helps when…	Pure reasoning may be better when…	Main evidence
Arithmetic / symbolic	Exactness matters and executable representation is available	The operation is trivial and tool latency dominates	PAL, Toolformer arXiv
Factual QA	Information is current, obscure, private, or citation-sensitive	The answer is stable, common, and low-stakes	ReAct, RAG arXiv
API workflows	The user needs an external operation or private system state	The user only needs conceptual explanation	Gorilla, API-Bank, BFCL arXiv+2ACL Anthology+2
Long-horizon tasks	Environment feedback is necessary	The task is a single static transformation	ReAct, BFCL arXiv
Large tool libraries	Tool search or retrieval can narrow the surface	Tool descriptions consume more context than the task deserves	Anthropic Tool Search discussion Anthropic

A good system does not “always use tools.” It uses tools when the expected value of external observation or execution exceeds the cost and risk of the call.

7. Failure modes

Tool use introduces new failure modes because it turns generation into a distributed system. Some failures are familiar language-model errors in structured form; others are unique to the model-tool boundary.

7.1 Tool hallucination

Tool hallucination occurs when a model invokes a nonexistent function, fabricates an API endpoint, invents an argument, or assumes a capability not exposed by the tool list. Gorilla’s motivation explicitly identifies hallucinated API usage and inaccurate input arguments as major problems even for strong models. arXiv

Mitigations include constrained tool lists, schema validation, retrieval over current documentation, native tool-choice controls, and negative examples. But tool hallucination is not eliminated by schemas alone: a model can select a valid tool for the wrong reason or fill valid fields with semantically wrong values.

7.2 Malformed arguments and schema mismatch

Malformed arguments include invalid JSON, wrong argument names, wrong types, missing required fields, and values outside allowed ranges. ToolScan identifies several related error categories: incorrect argument value, incorrect argument name, incorrect argument type, invalid format error, and incorrect function name. arXiv

Native APIs reduce but do not eliminate these failures. OpenAI’s strict mode and Structured Outputs make schema adherence stronger, but strict schemas also have constraints and may add initial latency or incompatibilities depending on schema shape. OpenAI Platform Google’s VALIDATED, ANY, and NONE modes similarly expose tool-calling control as a developer-level design choice rather than an automatic guarantee. Google Cloud Documentation

7.3 Ignored or misused tool outputs

A model can call the right tool and then ignore the result, misread it, overgeneralize from it, or blend it with stale parametric memory. This is especially common when tool outputs are long, poorly formatted, ambiguous, or contain irrelevant information. Anthropic’s tool-design guidance emphasizes that agents may process tool responses incorrectly and recommends optimizing response structure, truncation, pagination, and context-efficient formats. Anthropic

This failure mode is epistemically dangerous because the final answer may look grounded merely because a tool was used. A citation or tool trace is evidence only if the answer actually follows from the returned observation.

7.4 Repeated calls and infinite loops

Tool loops happen when a model repeats the same call, alternates between tools without progress, or keeps seeking more information despite having enough to answer. ToolScan includes repeated API calls as one of its error patterns. arXiv OpenAI’s Agents SDK includes a reset_tool_choice behavior, enabled by default, to avoid the model getting stuck in tool-use loops after a prior tool call. OpenAI GitHub

Mitigations include call budgets, loop detectors, “same arguments” suppression, maximum step counts, explicit stopping criteria, and controller-level policies that force synthesis after sufficient observations.

7.5 Over-tooling and tool-surface bloat

More tools do not automatically make an agent better. Anthropic’s engineering guidance warns that agents may choose wrong tools, pass wrong parameters, use too few tools, use too many tools, or process responses incorrectly, and recommends selectively implementing tools to reduce context and mistakes. Anthropic Large tool lists also create token overhead. Anthropic’s Tool Search work was motivated partly by the observation that several MCP servers can consume very large portions of context before task execution begins. Anthropic

Tool-surface bloat is a product-design problem, not just a model problem. Human API surfaces often assume deterministic callers with documentation-reading ability. Agentic tool surfaces should instead expose high-level affordances, narrow schemas, clear names, compact outputs, and task-relevant defaults.

7.6 Tool-result injection and security boundaries

Tool outputs are untrusted inputs. A web page, retrieved document, email, log file, or API response can contain instructions such as “ignore previous instructions” or “send secrets to this endpoint.” When the model treats tool outputs as part of its prompt, this becomes Prompt Injection through the environment.

The mitigation is architectural: separate data from instructions, sandbox tools, apply allowlists, require confirmation for side effects, avoid exposing secrets to the model unless necessary, validate outbound actions, and log tool traces. Native tool calling helps because the application remains the executor, but it does not make the content of tool results safe.

7.7 Side-effect errors

Read-only tools can produce wrong answers. Side-effectful tools can produce wrong worlds. A bad search query may waste time; a bad database update, email send, trade, deletion, or deployment can cause irreversible harm.

This is why tool use and agency diverge. A function-call API gives the model a way to request action; it does not decide which actions should require human approval, how to roll back operations, who is authorized, or which invariants must be maintained. Those are system-level responsibilities.

8. Tool use and agentic systems

Tool use is necessary but not sufficient for agency.

A calculator-using model is not automatically an agent. A search-using chatbot is not automatically an agent. Even a model that can call APIs is not necessarily agentic if it lacks persistent goals, state, planning, autonomy, and monitoring. Conversely, an agent almost always needs tools, because agency without environment interaction is only simulated agency.

OpenAI’s Agents SDK describes agents as applications that plan, call tools, collaborate across specialists, and keep state; its SDK runtime manages turns, tool execution, approvals, state, guardrails, handoffs, sessions, and orchestration. OpenAI Developers This is a useful decomposition. Tool calling is one layer. Agent runtime is another. Product policy, permissions, and evaluation are yet more layers.

A practical agent stack usually contains:

Layer	Responsibility	Tool-use relation
Model	Interpret instructions, decide actions, synthesize responses	Emits tool calls and integrates observations
Tool registry	Describe available affordances	Provides schemas, names, descriptions, examples
Executor	Actually run tools	Applies validation, auth, sandboxing, retries
State/memory	Preserve context across turns and tasks	Stores observations, plans, user preferences, artifacts
Controller/runtime	Manage loops and stopping conditions	Enforces budgets, step limits, handoffs, approvals
Guardrails	Prevent unsafe or invalid actions	Filters inputs, outputs, calls, and side effects
Evaluator/tracer	Measure trajectory quality	Records calls, latency, errors, cost, outcomes

ReAct can be read as the minimal agent skeleton: thought, action, observation, thought. But production agents require more than that skeleton. They need typed tools, state management, error handling, observability, security, and product-specific policies.

9. RAG as a special case of tool use

Retrieval-Augmented Generation is often treated as its own architecture, but it is more precise to say that RAG is a special case of tool use. The retriever is a tool; the query is the argument; the retrieved passages are the observation; the generator integrates the observation into an answer.

The original RAG formulation combined parametric memory with non-parametric memory accessed through a dense vector index, improving open-domain QA and producing more specific and factual generations than parametric-only baselines. arXiv Modern file search, enterprise search, database lookup, and web grounding are all variations on this pattern. Google’s Gemini docs explicitly describe File Search as enabling RAG, while OpenAI’s and Anthropic’s tool surfaces include retrieval and search-style tools alongside function calling and code execution. Google AI for Developers+2OpenAI Platform+2

RAG differs from other tools in four ways. First, it is usually read-only. Second, it is often automatically invoked by the system rather than explicitly selected by the model. Third, the result is usually natural language or document chunks, not a compact typed object. Fourth, the final answer depends heavily on ranking, chunking, citation alignment, and context-window management.

Those differences matter, but they do not make RAG conceptually separate. A RAG system is a tool-using language model whose main tool is a retriever.

10. Native tool calling versus prompt-level tool use

Native tool calling improves the engineering substrate, but it does not remove the need for prompt and system design.

Prompt-level tools have advantages: they are easy to prototype, model-agnostic, and transparent in text traces. ReAct-style prompting can be implemented around almost any model that follows instructions. But prompt-level parsing is brittle. The controller must parse text, recover from formatting errors, and distinguish reasoning from action. It also cannot always rely on the provider’s model-specific training for schema adherence.

Native tool calling has several advantages:

1. Typed contracts. Tools can be described with schemas rather than prose alone.

2. Execution separation. The model requests calls; the system executes them.

3. Validation. Invalid calls can be rejected before execution.

4. Observability. Tool calls, arguments, outputs, latency, and cost can be traced.

5. Control. Developers can force, forbid, or restrict tool use in many APIs.

6. Security boundaries. Applications can enforce authorization and side-effect policies.

But native APIs can also hide complexity. A tool call object may look clean while representing a semantically wrong choice. A strict schema may ensure valid JSON but not valid intent. A hosted tool may simplify implementation while making the provider’s execution semantics part of the application’s trust model.

The right mental model is not “native tool calling solves tool use.” It is “native tool calling makes the model-environment boundary explicit enough to engineer.”

11. Design principles for robust tool-using systems

11.1 Design tools for models, not humans

Human APIs often expose low-level operations because human developers can read documentation, compose calls, and handle edge cases. Models need affordances that are semantically direct. Anthropic’s tool-design guidance recommends selectively implementing tools, reducing unnecessary tools, improving descriptions, and using strict data models; it reports that tool-description refinements contributed to major benchmark improvements in their coding-agent work. Anthropic

A model-facing tool should have a clear name, narrow purpose, unambiguous parameters, compact output, and examples of correct use. The best tool is often not http_request; it is create_calendar_event, search_customer_invoices, or run_unit_tests.

11.2 Prefer high-level task tools over low-level primitives

Low-level primitives maximize flexibility but burden the model with planning. High-level tools encode domain invariants and reduce the number of calls. For example, refund_order(order_id, reason) is safer than exposing arbitrary database writes. search_policy_docs(query, jurisdiction) is safer than exposing raw vector-index parameters.

This is not always true. Code interpreters and shell tools are powerful precisely because they are general. But general tools should be sandboxed, budgeted, and audited more aggressively.

11.3 Separate tool selection from execution authority

The model may be allowed to propose actions it is not allowed to execute automatically. Read-only tools can often run without confirmation. Side-effectful tools should be gated by permission, policy, risk, and user intent. The execution layer should treat model output as a request, not as an order.

11.4 Validate and normalize arguments

Schema validation catches malformed calls, not all bad calls. A robust executor also normalizes dates, resolves entity IDs, checks authorization, validates preconditions, and rejects ambiguous or dangerous arguments. ToolScan’s error taxonomy shows that argument names, values, types, formats, and function names are all common failure points. arXiv

11.5 Budget loops explicitly

Every agent loop needs stopping conditions. Useful controls include maximum calls, maximum repeated calls, maximum wall-clock time, maximum cost, maximum retries, and forced synthesis after sufficient evidence. Loop control should live outside the model because the model is not a reliable judge of its own persistence.

11.6 Evaluate trajectories, not just final answers

A final answer can be correct despite a bad trajectory, and a trajectory can look plausible while producing a fragile answer. Tool-use evaluation should inspect tool selection, argument construction, execution success, result integration, latency, cost, and abstention. BFCL’s inclusion of serial and parallel calls, abstention, and stateful multi-step interactions is an example of evaluation moving beyond one-shot answer accuracy. Proceedings of Machine Learning Research

11.7 Treat retrieval outputs as evidence, not instructions

RAG and web search results should be treated as data. The model should use retrieved content to answer questions, not obey instructions embedded in that content. This distinction is central to secure tool use because many tool outputs originate from untrusted or adversarial environments.

12. The open architectural question: tool use or unified environment abstraction?

The field is moving toward a broader abstraction in which tool calls, retrieval, code execution, browsing, computer control, memory, and multi-agent handoffs are all environment interactions. OpenAI’s Agents SDK, Anthropic’s tool-search and programmatic tool-use work, and Google’s Interactions API all point in this direction: tools are no longer isolated function calls but components of observable, stateful, multi-step systems. OpenAI Developers+2Anthropic+2

Under the unified view, an LLM-based system resembles an interactive policy:

state/context -> model -> action/tool call -> environment transition -> observation -> model -> ...

This abstraction covers search, RAG, code execution, browser use, file editing, API workflows, and robot control. It also fits the language of agents, where the relevant unit is not a single response but a trajectory through an environment.

However, “tool use” remains a useful distinct concept for engineering. A tool is a boundary with a name, schema, executor, permission model, cost, and trace. Those boundaries matter for security, observability, debugging, evaluation, and product governance. Even if research converges on a model-plus-environment formalism, production systems still need named contracts.

The likely outcome is layered vocabulary. Researchers may increasingly speak in terms of environments, policies, observations, and trajectories. API providers may speak in terms of tools, functions, built-ins, hosted tools, MCP servers, and agent runtimes. Application engineers will need both: the environment abstraction for reasoning about behavior, and the tool abstraction for building safe systems.

The evidence for convergence is strong at the product-architecture level but thin at the scientific level. Vendors are clearly merging tools into agent runtimes and stateful interaction APIs. It is not yet settled whether this produces models that robustly learn general environment interaction, or whether it produces better wrappers around models that remain brittle in long-horizon settings. BFCL’s conclusion that memory, dynamic decision-making, and long-horizon reasoning remain open challenges is a useful corrective to overclaiming. Proceedings of Machine Learning Research

13. Summary claims

Tool use became central because language models are powerful interpreters of intent but unreliable substitutes for calculators, search engines, databases, code runtimes, and external systems. MRKL supplied the modular architecture, ReAct supplied the interleaved reasoning-action-observation loop, PAL supplied the executable-program pattern, Toolformer supplied self-supervised API-call learning, and Gorilla reframed API usage as a documentation-grounded call-generation problem.

Native tool-calling APIs turned these research patterns into deployable interfaces. OpenAI, Anthropic, and Google now expose structured mechanisms for declaring tools, receiving tool calls, executing them externally or through hosted systems, and returning results into the model’s context. The APIs differ, but the shared contract is stable: the model proposes; the executor validates and runs; the model integrates.

Empirically, tool use helps most when the external system is more reliable than the model for the relevant operation: exact computation, current lookup, private data access, executable code, and side-effectful workflow automation. It adds overhead when the answer is simple, stable, or cheaper to produce directly; when the tool surface is too large; when outputs are too verbose; or when execution introduces avoidable latency and failure modes.

The main failures are not mysterious. Models call nonexistent tools, choose wrong tools, produce malformed arguments, ignore outputs, repeat calls, overuse tools, and become vulnerable to untrusted observations. Native schemas, strict modes, tool search, retrieval over documentation, controller budgets, sandboxing, and trajectory-level evaluation reduce these problems but do not eliminate them.

Tool use is therefore a necessary substrate for agentic systems, not a sufficient condition for agency. The future likely merges tool use into a broader model-environment abstraction, but named tools will remain important wherever developers need typed contracts, permissions, observability, and accountability.

References and primary sources

• Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools” (2023). Introduces self-supervised API-call learning for tools such as calculators, search engines, translation systems, and calendars. arXiv

• Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” (2022). Defines the interleaved reasoning-action-observation prompting pattern and evaluates it on QA and interactive tasks. arXiv

• Karpas et al., “MRKL Systems: A Modular, Neuro-Symbolic Architecture” (2022). Frames language models as part of a modular system with neural and symbolic expert modules. arXiv

• Gao et al., “PAL: Program-Aided Language Models” (2022). Shows how generating executable programs and running them in Python can outperform prose-only reasoning on math and symbolic tasks. arXiv

• Patil et al., “Gorilla: Large Language Model Connected with Massive APIs” (2023). Studies accurate API-call generation and retrieval-grounded mitigation of hallucinated APIs. arXiv

• OpenAI, “Function calling and other API updates” (2023), Function Calling docs, Structured Outputs docs, and Agents SDK docs. Primary documentation for OpenAI native tool calling, strict schemas, hosted tools, and agent runtimes. OpenAI Developers+3OpenAI+3OpenAI Platform+3

• Anthropic, Claude Tool Use docs and engineering posts. Primary documentation for tool_use, tool_result, client/server tools, tool-design guidance, tool search, and programmatic tool calling. Anthropic+3Claude Platform+3Claude Platform+3

• Google, Gemini Function Calling, Tools, Vertex AI modes, and Interactions API docs. Primary documentation for Gemini tool declarations, built-in tools, function-calling modes, context circulation, and stateful interaction APIs. Google AI for Developers+3Google AI for Developers+3Google AI for Developers+3

• API-Bank, ToolLLM/ToolBench, BFCL, and ToolScan. Benchmarks and analyses for tool-use dialogues, large API surfaces, native function-calling evaluation, and error taxonomies. arXiv+3ACL Anthology+3arXiv+3

• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Primary RAG paper framing generation as a combination of parametric and non-parametric memory. arXiv

Companion entries

Core theory: Tool Calling, Tool-Augmented Language Models, ReAct, MRKL Systems, Program-Aided Language Models, Toolformer, Model + Environment Abstraction, Neuro-Symbolic AI

Agent systems: Agentic Systems, Agent Runtime, Planning and Acting, Multi-Agent Handoffs, Human-in-the-Loop Oversight, Autonomy Levels in AI Systems

Retrieval and grounding: Retrieval-Augmented Generation, File Search, Web Grounding, Vector Databases, Citation Faithfulness, Epistemic Dependence on Tools

Engineering practice: Function Calling APIs, Structured Outputs, JSON Schema for LLMs, Model Context Protocol, Code Interpreter, Tool Schema Design, Agent Eval Harnesses

Failure modes and safety: Tool Hallucination, Malformed Tool Arguments, Ignored Tool Outputs, Agentic Loop Failures, Prompt Injection, Side-Effect Safety, Tool Result Sandboxing

Benchmarks: APIBench, API-Bank, ToolBench, Berkeley Function Calling Leaderboard, ToolScan, WebShop, ALFWorld