Learning map / Task 1

Build an agent from scratch, then make it ask the model less

One order desk, one question, four versions of the program that answers it, each cheaper to run than the last. By the end you can state, from a trace (the written record of one run), what each request to the model costs and which change to the program removes it.

Reading: about 25 minutes, no programming needed. Building the four versions yourself, for readers who write Python: an afternoon and an API key (the password your program sends with each request).

• API
• software engineering
• production and scale
• Python

The scenario, toy to production

A support desk asks: "For customers C1 to C4, list open orders and the shipping status of each." The data lives in tasks/t01_fewer_tool_calls/store.py. Today it is three customers in a small lookup table in the code (a Python dictionary), and the question names a fourth. The same loop will be asked to do ten thousand customers nightly, and every extra request to the model then costs time and money, and a request that fails costs them again when it is sent again .

The data. Two tools do the work. get_orders takes a customer and returns that customer's open orders, the ones not delivered yet, each with an order number. get_status takes one order number and returns its shipping status, so no status can be asked for until the orders are known. Open orders per customer in the question:

So the question is 2 + 1 + 3 open orders, 6 statuses to look up. C4 is not a customer, so looking it up fails. The model does not know that until it asks, and in this lesson it never asks again: C4 does not exist, so a retry could not help.

Six more words, used on every line below:

• A round trip is one request your code (the server code: the program a developer, maybe you, writes; not the person at the desk) sends to the model and the one reply it gets back. The model cannot act between requests; everything it learns arrives in the next one.
• A tool call is one ask the model makes inside a reply (not a request: only what your code sends the model is a request), asking your code to run one tool once, such as "look up the orders of C1".
• Parallel tool use is one reply holding several tool calls at once.
• A tool result is your code's answer to one tool call. Your code sends one per call, in the next request.
• Tokens are the pieces of text you pay for: a word or part of a word each. Every count on this page comes from this lesson's stand-in for Claude, a scripted program that follows the documented tool-use rules (How to count, below); it counts characters divided by four, so a count here is an estimate of the shape, not a bill. The counts on this page are what is sent (input tokens); the replies' own tokens are billed too.
• A trace is the written record of one run: every request, reply and tool call in order, with its counts.

Three costs move separately: round trips (waiting, and the whole conversation so far, the transcript, re-read each time), tool calls (your code's work), and tokens .

One round trip, and why each one costs more than the last

A picture to hold on to. Think of the model as an expert you can reach only by phone. They cannot see your systems, and they forget each call the moment it ends. So every time you call, you read back the whole conversation from the start. Then they either give the answer or ask you to look things up: "check C1's orders, and C2's". You hang up, look them up, and call back with everything said so far plus what you found. Each call is a round trip. Each lookup they ask for is a tool call. The read-back is literal: the API keeps no memory between requests, so your code resends the whole conversation every time .

The conversation is a list of messages, each labeled with who sent it: assistant for the model's replies, and user for everything your code sends, tool results included . So a "user message" on this page is your code speaking, not the support-desk person.

Not a message to the API. You type the question; the server code builds request 1 from it.
the question: "For customers C1, C2, C3 and C4: list each customer's open orders, then the shipping status of every order. Finish with one line per customer."

user message (173 characters as sent, its labels and quotes included):
  text: "For customers C1, C2, C3 and C4: list each customer's open orders, then the shipping status of every order. Finish with one line per customer."
tools offered (432 characters, the input schema of each included):
  get_orders: "Open orders for one customer."
  get_status: "Shipping status of one order."
the rest of the request (75 characters): the model name, the limits, the field names and the commas between messages
characters: 173 + 432 + 75 = 680; divided by four, rounded down: 170 input tokens

stop_reason: tool_use
  tool_use toolu_0001_get_orders: get_orders({"customer_id":"C1"})
  tool_use toolu_0002_get_orders: get_orders({"customer_id":"C2"})
  tool_use toolu_0003_get_orders: get_orders({"customer_id":"C3"})
  tool_use toolu_0004_get_orders: get_orders({"customer_id":"C4"})

sent back in request 2, one tool_result per call:
  tool_result for toolu_0001_get_orders: "{"customer_id": "C1", "orders": ["ord-101", "ord-102"]}"
  tool_result for toolu_0002_get_orders: "{"customer_id": "C2", "orders": ["ord-201"]}"
  tool_result for toolu_0003_get_orders: "{"customer_id": "C3", "orders": ["ord-301", "ord-302", "ord-303"]}"
  tool_result for toolu_0004_get_orders is_error=true: "'C4' not found"

user message (173 characters as sent, its labels and quotes included):
  text: "For customers C1, C2, C3 and C4: list each customer's open orders, then the shipping status of every order. Finish with one line per customer."
assistant message (462 characters as sent, its labels and quotes included):
  tool_use toolu_0001_get_orders: get_orders({"customer_id":"C1"})
  tool_use toolu_0002_get_orders: get_orders({"customer_id":"C2"})
  tool_use toolu_0003_get_orders: get_orders({"customer_id":"C3"})
  tool_use toolu_0004_get_orders: get_orders({"customer_id":"C4"})
user message (576 characters as sent, its labels and quotes included):
  tool_result for toolu_0001_get_orders: "{"customer_id": "C1", "orders": ["ord-101", "ord-102"]}"
  tool_result for toolu_0002_get_orders: "{"customer_id": "C2", "orders": ["ord-201"]}"
  tool_result for toolu_0003_get_orders: "{"customer_id": "C3", "orders": ["ord-301", "ord-302", "ord-303"]}"
  tool_result for toolu_0004_get_orders is_error=true: "'C4' not found"
tools offered (432 characters, the input schema of each included):
  get_orders: "Open orders for one customer."
  get_status: "Shipping status of one order."
the rest of the request (79 characters): the model name, the limits, the field names and the commas between messages
characters: 173 + 462 + 576 + 432 + 79 = 1,722; divided by four, rounded down: 430 input tokens
the figure splits the same 430 by the request that first sent each part: 170 + 260, where each new part is this request's count minus the one before

stop_reason: tool_use
  tool_use toolu_0005_get_status: get_status({"order_id":"ord-101"})
  tool_use toolu_0006_get_status: get_status({"order_id":"ord-102"})
  tool_use toolu_0007_get_status: get_status({"order_id":"ord-201"})
  tool_use toolu_0008_get_status: get_status({"order_id":"ord-301"})
  tool_use toolu_0009_get_status: get_status({"order_id":"ord-302"})
  tool_use toolu_0010_get_status: get_status({"order_id":"ord-303"})

sent back in request 3, one tool_result per call:
  tool_result for toolu_0005_get_status: "{"order_id": "ord-101", "status": "shipped"}"
  tool_result for toolu_0006_get_status: "{"order_id": "ord-102", "status": "packed"}"
  tool_result for toolu_0007_get_status: "{"order_id": "ord-201", "status": "shipped"}"
  tool_result for toolu_0008_get_status: "{"order_id": "ord-301", "status": "delayed"}"
  tool_result for toolu_0009_get_status: "{"order_id": "ord-302", "status": "shipped"}"
  tool_result for toolu_0010_get_status: "{"order_id": "ord-303", "status": "packed"}"

user message (173 characters as sent, its labels and quotes included):
  text: "For customers C1, C2, C3 and C4: list each customer's open orders, then the shipping status of every order. Finish with one line per customer."
assistant message (462 characters as sent, its labels and quotes included):
  tool_use toolu_0001_get_orders: get_orders({"customer_id":"C1"})
  tool_use toolu_0002_get_orders: get_orders({"customer_id":"C2"})
  tool_use toolu_0003_get_orders: get_orders({"customer_id":"C3"})
  tool_use toolu_0004_get_orders: get_orders({"customer_id":"C4"})
user message (576 characters as sent, its labels and quotes included):
  tool_result for toolu_0001_get_orders: "{"customer_id": "C1", "orders": ["ord-101", "ord-102"]}"
  tool_result for toolu_0002_get_orders: "{"customer_id": "C2", "orders": ["ord-201"]}"
  tool_result for toolu_0003_get_orders: "{"customer_id": "C3", "orders": ["ord-301", "ord-302", "ord-303"]}"
  tool_result for toolu_0004_get_orders is_error=true: "'C4' not found"
assistant message (688 characters as sent, its labels and quotes included):
  tool_use toolu_0005_get_status: get_status({"order_id":"ord-101"})
  tool_use toolu_0006_get_status: get_status({"order_id":"ord-102"})
  tool_use toolu_0007_get_status: get_status({"order_id":"ord-201"})
  tool_use toolu_0008_get_status: get_status({"order_id":"ord-301"})
  tool_use toolu_0009_get_status: get_status({"order_id":"ord-302"})
  tool_use toolu_0010_get_status: get_status({"order_id":"ord-303"})
user message (819 characters as sent, its labels and quotes included):
  tool_result for toolu_0005_get_status: "{"order_id": "ord-101", "status": "shipped"}"
  tool_result for toolu_0006_get_status: "{"order_id": "ord-102", "status": "packed"}"
  tool_result for toolu_0007_get_status: "{"order_id": "ord-201", "status": "shipped"}"
  tool_result for toolu_0008_get_status: "{"order_id": "ord-301", "status": "delayed"}"
  tool_result for toolu_0009_get_status: "{"order_id": "ord-302", "status": "shipped"}"
  tool_result for toolu_0010_get_status: "{"order_id": "ord-303", "status": "packed"}"
tools offered (432 characters, the input schema of each included):
  get_orders: "Open orders for one customer."
  get_status: "Shipping status of one order."
the rest of the request (83 characters): the model name, the limits, the field names and the commas between messages
characters: 173 + 462 + 576 + 688 + 819 + 432 + 83 = 3,233; divided by four, rounded down: 808 input tokens
the figure splits the same 808 by the request that first sent each part: 170 + 260 + 378, where each new part is this request's count minus the one before

stop_reason: end_turn
  text: "C1: 2 open orders (ord-101 shipped, ord-102 packed) C2: 1 open orders (ord-201 shipped) C3: 3 open orders (ord-301 delayed, ord-302 shipped, ord-303 packed) C4: could not be found"

Not a message to the API. The server code prints the final reply's text; that is the list you see.
the answer: "C1: 2 open orders (ord-101 shipped, ord-102 packed) C2: 1 open orders (ord-201 shipped) C3: 3 open orders (ord-301 delayed, ord-302 shipped, ord-303 packed) C4: could not be found"

"stop_reason": "tool_use",
"content": [
  {"type": "tool_use", "id": "toolu_1", "name": "get_orders", "input": {"customer_id": "C1"}},
  {"type": "tool_use", "id": "toolu_2", "name": "get_orders", "input": {"customer_id": "C2"}}
]

{"role": "user", "content": [
  {"type": "tool_result", "tool_use_id": "toolu_1", "content": "[\"ord-101\", \"ord-102\"]"},
  {"type": "tool_result", "tool_use_id": "toolu_2", "content": "[\"ord-201\"]"}
]}

What you need to predict

Hint for v0. The model's first reply is parallel, as always. Claude's docs say results split across several messages teach the model to avoid parallel calls . This lesson's scripted model takes that to the limit: once the results of a parallel reply come back split, it asks for one thing per reply from its next reply on. Predict v1 first, then ask what that rule changes for v0 after its first reply.

Predict before you run

Rungs 0 to 3 have worked versions under worked/; rung 4 has none. Work each one out from what the page states, write your numbers, then reveal. A wrong prediction is still the useful outcome: it names the rule you did not yet hold, and the reveal draws every rung the way v1 is drawn above, so you can see where your count and the measured one part ways.

Reveal measured traces How to run them

make install
make test        # prints every rung's trace with -s; the numbers come from rungs.json
make practice    # your target; red until agent.py meets it

The ladder

The ladder is the hands-on part, for readers who write Python. If you do not program, everything it builds is explained in the answers after the reveal; skip ahead to the interview questions.

You build in tasks/t01_fewer_tool_calls/learner/agent.py, a copy of rung 0. Run make install once. make practice runs your file against rung 3's target and stays red until it meets it; make test prints every worked rung's trace to compare with.

Five-minute lessons

Teach it back

Write tasks/t01_fewer_tool_calls/learner/explain.md: what you changed, why each change moved which cost, and when you would not make it. Then make feedback (it needs ANTHROPIC_API_KEY, like make live). Every finding you get back cites the catalog (web/resources.json, the list of sources this page draws on) or is dropped and counted, so you can check each one yourself.

Interview questions this task prepares

1. Walk me through one turn of a tool-use loop. What must the next request contain?
   A model answer

   Your code sends the whole conversation and the tool list, because the API keeps no memory between requests . If the reply's stop reason is tool_use, it holds one or more tool calls . Your code runs each one. The next request is everything so far, the model's reply, then one user message with a tool result per call, matched by id, placed before any text, failures marked is_error . Repeat until the stop reason is end_turn.

2. Your agent makes forty calls for a task that needs four. Where do you look first?
   A model answer

   At a trace, counting round trips and tool calls separately, since they have different causes. Then four suspects, cheapest first. Results split across user messages, which teaches the model to stop calling in parallel . Tools that take one item when a list tool fits, as v1 against v2 shows. Failures not marked is_error, the documented way to say a tool failed, or error messages that do not say what to try next . Tool descriptions that leave the model unsure which tool fits .

3. When is a coarse tool the wrong design?
   A model answer

   When the questions vary, because a tool shaped for one question needs a sibling for every variant. When the model should judge the in-between results, which a coarse tool hides. When its result would be too large to send whole . And when other tasks reuse the small tools. The guidance favors consolidating ; these exceptions are reasoning, so say so in the interview.

4. What is the difference between fewer round trips and fewer tokens? Give a case where they move apart.
   A model answer

   Round trips count waits; tokens count text, which is what you pay for. v1 to v2 kept 3 round trips and cut tokens, because 2 calls and results replaced 10. The other way round: v3 has fewer round trips than v2, but if its single result carried every order's full history, tokens could rise. Prompt caching is a third case: it cuts token cost and leaves round trips alone .

5. How would you measure whether a tool change helped?
   A model answer

   Run the same set of scenarios before and after, and record round trips, tool calls, tokens and whether the answer was right, per completed task rather than per request . This task's own tests do that for one scenario; a real check uses many more, twenty in the production rung, because one scenario can flatter a change.

6. What does is_error change about what the model does next?
   A model answer

   It tells the model the tool failed, rather than returned data. The docs say Claude works the error into its answer, and retries with corrections when its own input was invalid; write the error so it says what went wrong and what to try next . The docs do not say what happens to an unflagged error, so a candid answer says that too.