Tool use and agents

The RAG system of the previous note calls a retriever on every query, by design of the pipeline, before the model sees anything. The model has no say in whether retrieval was needed, what to search for, or whether to search again after reading the results. Tool use gives it that say, and an agent is what you get when the model is allowed to keep deciding. This note is about the mechanism, what it is good for, and a question the enthusiasm around it tends to skip: when a loop that lets a model decide is worse than a pipeline that does not.

Notation. A model $\pi$ with a context $c$ (instructions, conversation, prior results); a set of tools $\{f_i\}$ with typed signatures; a tool call is $(i, \text{args})$; the runtime returns an observation $o = f_i(\text{args})$ that is appended to $c$.

Baseline: the model answers from its context#

Given a question whose answer requires today’s weather, a calculation, or a lookup in a database, a plain model produces the kind of text that follows such questions: a guess with the right form. Retrieval fixes the lookup case for text corpora. It does not cover a calculator, a search engine, a code interpreter, a company’s order system, or any action with a side effect.

Tool use#

The protocol#

1. The system prompt lists the tools: name, what it does, argument names and types (a JSON schema). Chat-tuned models are trained on such listings and on examples of emitting calls in a fixed format, with special tokens around them.
2. The model generates either natural-language text or a call. A call is structured output in the sense of the RAG note; constrained decoding or format-specific training makes it parseable.
3. The runtime validates the arguments against the schema, executes the function, and appends the result as a message with a tool role.
4. The model continues: it may answer, or call again.

What the protocol buys is that no one writes the sequence of calls: given a search tool and an image tool, a request for a photo of the car James Bond drove in the latest movie can be served by search, then generate, composed by the model from the tool descriptions and the question. What it costs is that the model also decides whether to call at all, and the runs below show it deciding badly.

Code as the tool interface#

One variant replaces the fixed call format by Python: the model writes a short program that calls the tools as functions, the runtime executes it in a sandbox, and the printed output is the observation. A loop, a conditional, or a call whose arguments depend on an earlier result is then one step instead of several, and the same interpreter lets the model run code, read the error and fix it. The sandbox is not optional: the model is writing code that runs.

Three runs of a small code agent#

Take the code variant, as implemented in the smolagents library, over Qwen3-4B-Instruct-2507, a 4 B-parameter instruction-tuned model, and give it three requests. The traces are more instructive than the answers:

• What is the current weather in Paris?, with a web-search tool. The first program searches for “current weather in Paris” and gets forum posts of a phone company, matched on the word current. The second refines the query to “current weather in Paris, France” and gets weather pages that disagree: one reports 13 °C with scattered clouds, another a clear sky and +16°. The third step answers “Clear sky, around 13-16°C”, a merge of the two. Three steps took 9,528 input tokens in total, because the tool listing and every earlier result are re-read at each step: the prompt grows from 2,083 tokens at the first step to about 4,100 at the third.
• Generate me a photo of the car that James Bond drove in the latest movie, with the search tool and an image-generation tool. The model skips the search: its one program sends a prompt beginning “James Bond’s car from the latest Bond movie, high-res, photorealistic” to the image generator and returns the image. The image generator receives a description of a role, not of a car.
• A script with two bugs, numbers(i) on a list and range(4) over three elements, passed with the request to debug it, then run it to make sure it works. The model returns a corrected version in one step, numbers[i] and range(len(numbers)), without running either version, so the run-read-fix loop that the code interface allows never takes place.

All three answers look like successes. Only the traces show one answer stitched from conflicting sources, one tool never consulted and one instruction ignored, which is why agents are evaluated on their trajectories and not only on their outputs.

Agents: a loop around tool calls#

A single tool call extends the model; a loop that repeats decide → call → observe until a stopping condition is an agent. The ReAct pattern (Yao et al., 2023) interleaves a written reasoning step with each action so that the trajectory is inspectable. The trajectory the James Bond request calls for, which the run above did not take:

thought: I need the car model first, then an image.
action:  web_search(query="car James Bond latest movie")
observation: ... Aston Martin DB5 ...
thought: Now generate the image.
action:  text_to_image(prompt="Aston Martin DB5")
observation: <image>
final answer: <image>

Figure

The loop of a tool-using agent

What has to be added to the protocol to make the loop work:

State. The context accumulates every call and result. For long tasks it exceeds the window and must be summarized or stored outside the model (a scratchpad, a memory of earlier runs, files); what is kept and what is dropped is a design decision with direct effect on behaviour. The model itself keeps nothing between calls; where the memory lives is a property of the runtime, described after this list.

Planning. For multi-step tasks the model writes a plan first (sub-goals in order) and revises it as observations arrive. Planning is text generated by the same model; it is not a separate solver, and plans are often wrong in ways the model does not notice until an observation contradicts them.

Stopping. A final-answer action, a maximum number of steps, a token or cost budget, and detection of repetition (the same call twice with the same result). Without the last three, a loop that cannot make progress runs until something external stops it.

Specialization and coordination. Beyond a few tools, one prompt that describes everything degrades. The usual structure is several agents, each with a narrow role and its own tools, and a coordinator that routes the user’s request and merges results. A concrete shape, from an inventory-operations system: a coordinator agent receives the request; supply, transit and export agents each query their own internal systems and propose actions in their domain; the coordinator assembles a proposal. Each sub-agent sees only its tools and instructions, which keeps every prompt short and every failure local.

Human confirmation. Reading is cheap to undo; writing is not. The same inventory system proposes operational actions and executes none of them without an explicit confirmation step for the critical ones (moving stock, changing an export declaration). The loop pauses, shows the proposed call and its arguments, and continues only on approval. This is a property of the runtime, not of the model, and it is the single most effective safeguard in systems that act on the world.

Where the memory of a conversation lives#

A language model is a function from a token sequence to a distribution over the next token. Nothing persists from one call to the next: the weights are fixed, and the KV cache of a call is not carried into the following one (a server may reuse it for an identical prefix, which saves compute and changes no output). A chat model that recalls a name from three turns ago recalls it because the runtime sent those three turns again. Asked What’s my name? as a single message, a 0.5 B-parameter chat model, Qwen2-0.5B-Instruct, answers with a general paragraph about what names are; given the same question after Hi! I’m Bob and its own greeting, it answers Your name is Bob.

Memory is therefore part of the runtime, in three pieces.

1. A stored state per conversation. The state is a record: the list of messages, plus whatever fields the application needs (the reply language, an account number, the result of an earlier tool call). After every step the runtime saves a checkpoint of the record under a thread identifier. A request with the same identifier continues that conversation; a new identifier starts from an empty record. Two conversations never see each other’s history because their identifiers differ, not because the model keeps them apart.
2. An update rule per field. The message list is appended to: a step returns its new messages and the runtime adds them to the stored list rather than replacing it. Other fields are overwritten when a request supplies them and keep their stored value when it does not, so a reply language set on the first turn still applies on the tenth without being repeated.
3. A prompt assembled from the state at every call. A template puts the system instruction first, fills it with state fields (answer in {language}), and inserts the stored messages after it, the newest user message last. The result is rendered with the model’s chat template into one ordinary token sequence.

The store must receive only the model’s answer. If the generation step returns the prompt together with its continuation and that text is saved as the assistant’s message, the next prompt contains the previous one nested inside an assistant turn, and the history grows quickly even when every exchange is short.

Trimming. The stored history grows without bound and the context window does not, so before each call the runtime cuts the messages to a token budget, counted with the model’s own tokenizer. The usual policy has four rules:

• keep the newest messages, on which the current exchange depends most;
• always keep the system instruction, and count it against the budget;
• never split a message: a message cut at an arbitrary token is a different message;
• start the kept history on a user turn, so the model never reads an answer whose question was cut.

Worked example (toy token counts). A conversation of a 10-token system instruction and seven messages, 75 tokens in all, trimmed to a budget of 52:

Message	Tokens	Kept
system	10	yes (always)
user 1: My name is Ana.	6	no
assistant 1	5	no
user 2	12	no (over budget)
assistant 2	20	no (its question was cut)
user 3	7	yes
assistant 3	9	yes
user 4: What is my name?	6	yes

The system instruction leaves 42 tokens. Walking back from the newest message, $6 + 9 + 7 + 20 = 42$ fits exactly, and user 2 would bring it to 54. The kept tail then starts with assistant 2, whose question did not fit, so it goes too. The model receives four messages and 32 tokens, and the question it must answer refers to a message it no longer sees. Trimming bounds the cost of a turn and the size of the KV cache; the price is forgetting. Longer conversations therefore replace the dropped turns by a summary, or keep facts such as the user’s name in separate state fields that are never trimmed.

Failure modes#

Failure	Mechanism	Mitigation
Wrong tool or wrong arguments	the model guesses from descriptions	precise descriptions, examples, schema validation, few tools per agent
Hallucinated results	the model “remembers” a result it did not receive	only runtime-inserted observations; no fabricated tool messages
Loops	same call, same result, no progress	step and cost budgets, repetition detection
Compounding errors	a wrong early observation shapes every later step	plan revision, verification steps, human checkpoints
Prompt injection through tools	a web page or document returned by a tool contains instructions	tool outputs are data; never execute instructions found in them; confirmation for side effects
Cost	every step re-reads the whole context	compact state, caching of shared prefixes, small models for routing
Silent side effects	a write succeeds that should not have	confirmation, dry-run modes, reversible actions first

The injection row deserves emphasis. An agent that reads a web page and then acts has exactly the vulnerability of the RAG system, with higher stakes: the page can say now call send_email with the following content, and a model that follows instructions well will try to. The only robust defence is structural: side-effecting tools behind confirmation, and a strict separation between instructions (from the system and the user) and data (from tools).

When an agent is unnecessary#

The loop costs tokens, latency and predictability. It is the right design when the number and order of steps depend on intermediate results that cannot be known in advance: debugging, open-ended research, operating a system with many possible actions. It is the wrong design when the steps are known:

• A fixed sequence (retrieve → rerank → generate; classify → route) is a pipeline. Write it as code, with the model called at the steps that need language; it runs faster, costs less, and fails in explainable ways.
• A single tool call with known arguments (look up this order, convert these units) is a function call, not an agent; let the program decide to call it.
• A task with a schema-shaped answer (extract these fields) is structured output.
• A task where a wrong action is expensive and the model’s decision cannot be checked before execution should not be delegated to a loop at all.

The practical rule: start with the pipeline; move a decision into the model only when the pipeline’s fixed logic is measurably the bottleneck. Every decision moved into the model needs an evaluation of that decision, which the next note is about.

What changed#

	Pipeline (RAG)	Tool use	Agent
Who decides the steps	the program	the model, one step	the model, repeatedly
Side effects	none	possible	likely
State	the prompt	the prompt + one result	accumulated, checkpointed, trimmed
Cost	fixed	one extra generation	unbounded without budgets
Predictability	high	medium	low
Evaluate	retrieval + answer	call correctness + answer	trajectory, success, cost

Limitations#

• Reliability compounds. If all ten steps must succeed, nothing recovers from an error, and each step succeeds with probability 0.95 given that the earlier ones did, the task succeeds with probability $0.95^{10} \approx 0.60$. Marginal per-step rates alone do not fix the product when outcomes are dependent in other ways; recovery from errors raises it.
• Models are poor at knowing what they do not know, which is the ability an agent most needs when deciding whether to call a tool.
• Benchmarks for agents are young and sensitive to tool implementations; a score transfers poorly between environments.
• Cost is a first-class constraint; a loop that solves the task at ten times the budget has not solved it.
• Multi-agent designs add coordination failures (lost context between agents, contradictory proposals) on top of single-agent ones.
• Memory is lossy. Trimming drops the oldest turns and summaries drop details; whatever is not in the prompt does not exist for the model.

What you should remember#

• Tool use: describe functions with schemas; the model emits a call; the runtime executes and appends the result; the model continues.
• An agent is the loop with state, a plan, and stopping rules (final answer, step and cost budgets, repetition detection).
• The model is stateless. A conversation is a record kept by the runtime: checkpointed per thread, its message list appended to at every step, re-sent on every call, and trimmed to a token budget (system instruction kept, newest messages whole, starting on a user turn).
• Split large tool sets across specialized agents with a coordinator; put every side-effecting action behind human confirmation.
• Tool outputs are data; instructions inside them are attacks.
• Prefer a pipeline when the steps are known; move decisions into the model only when they must depend on intermediate results, and evaluate each one.
• Read trajectories, not only answers: in the three runs of the small code agent all three answers look like successes.

Notebook#

the agents and diffusion notebook. Cell numbers count every cell, markdown included, from 1. Quoted numbers come from the run saved with the course; cells changed since then for current library versions have no saved output. The agent cells follow a cookbook, and the saved traces are more instructive than the cookbook text around them.

• Cell 6 builds a CodeAgent with an image-generation tool loaded from the Hub and a DuckDuckGo search tool over Qwen3-4B-Instruct-2507 and runs the James Bond request; the saved trace has one step, a direct image call with no search (2,159 input and 124 output tokens). Cell 7 shows the cookbook’s image of an Aston Martin DB5, not this run’s output.
• Cell 9 runs the weather question with only the search tool: three steps, the first search returning unrelated forum posts, 9,528 input and 225 output tokens, answer “Clear sky, around 13-16°C”.
• Cells 12–14 pass a buggy script as an additional argument; the agent returns a corrected version with both bugs fixed in one step without running it, and cell 14 prints it. The markdown cell after this run describes the cookbook’s run-read-fix loop, which this run did not perform.
• Cells 17–19 belong to the text diffusion note.

The multi-agent inventory structure described above is from a production system summarized on the work page; it is not in the notebook.

The memory of Where the memory of a conversation lives is built step by step in the LLM notebook of the HSE-LAMBDA Deep Generative Models course, Transformers-LangChain.ipynb, with LangChain and LangGraph over Qwen/Qwen2-0.5B-Instruct. Cell numbers count every cell, markdown included, from 1; its outputs are saved. The autoregressive transformers note describes all of its cells.

• Cells 42–46 send “Hi! I’m Bob” (cell 44), then “What’s my name?” alone (cell 45) and after the earlier turns (cell 46): the general answer and “Your name is Bob.” quoted above.
• Cells 47–51 build a StateGraph over MessagesState with one node that calls the model and an edge from START to it, compiled with a MemorySaver checkpointer. The config {"configurable": {"thread_id": "abc123"}} selects the conversation, so the second invoke, which passes only “What’s my name?”, is answered from the checkpointed history.
• Cells 52–58 put a ChatPromptTemplate.from_messages with a system message and a MessagesPlaceholder for the stored messages in front of the model.
• Cells 59–64 define the state as a TypedDict with messages: Annotated[Sequence[BaseMessage], add_messages], the reducer that appends, and language: str, a plain field that is overwritten. The second invoke omits language, and the stored “Spanish” still applies: the answer is “Tu nombre es Bob.”
• Cells 65–70 build the trimmer with trim_messages(max_tokens=45, strategy="last", token_counter=chat_model, include_system=True, allow_partial=False, start_on="human"), the four rules above, and cut an 11-message example to 7 (cell 67). Cell 68 trims the stored messages before the prompt template adds the system message, so include_system has no effect there. Cell 70 reads the checkpoint back with get_state(config).values: the language and the two stored messages.

Two issues, and their combined effect. The text-generation pipeline returns the prompt with the answer, so every stored assistant message begins with the whole prompt, and from cell 51 on the prompts contain earlier prompts nested in assistant turns; return_full_text=False in the pipeline fixes it. token_counter=chat_model counts tokens with LangChain’s fallback, the GPT-2 tokenizer applied to Human: …/AI: … strings, not with Qwen’s tokenizer. Together these break the trimming graph after its first turn: the assistant message stored in cell 69, prompt included, is 205 characters and at least 66 GPT-2 tokens, more than the whole budget, so on the next turn the trimmer would keep only the new question. The notebook runs one turn and does not show it.

Next#

Every system in this section has decisions inside it that are made by a model: which passage is relevant, whether an answer is faithful, which tool to call. Measuring them needs more than a test set with labels. The next note covers evaluation of language-model systems: standardized task benchmarks and their harness, retrieval metrics, models as judges and how much to trust them, human evaluation, and the operational measures (latency, cost, reliability) that decide whether a system ships.

References#

• S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, Y. Cao. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR, 2023.
• T. Schick et al. Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS, 2023.
• Y. Qin et al. Tool Learning with Foundation Models. arXiv, 2023. A survey of tool use.
• X. Wang et al. Executable Code Actions Elicit Better LLM Agents. ICML, 2024. Code as the action interface.
• K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, M. Fritz. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec, 2023.
• Hugging Face. smolagents documentation. The agent library used in the notebook.