Tool Calling, Explained: Why My Portfolio Chat Doesn't Use It

The chat on this site never calls a tool

Open /ask and ask it something about my background. Under the hood, the request does exactly four things: embed your question, pull the five closest chunks out of a static JSON file, glue them into the system prompt, and stream a response. No tools array. No back-and-forth. One pass, done.

That's not an accident — I wrote about why RAG beat long context for a corpus this small. But it means my own chat is a bad example of the thing people call an "AI agent." It doesn't decide anything mid-request. It doesn't call anything. It answers with whatever got glued in ahead of time.

So what does an assistant that actually calls tools do differently? I wanted to draw the mechanism, not just wave at the word "agent."

---

What "tool calling" means

A language model is, underneath everything, a function that predicts the next chunk of text. It cannot open a file, hit an API, or query a database — it can only produce tokens. Tool calling is the convention that lets it act anyway: the app hosting the model sends along a menu of tools (a name, a description, and a schema — a strict spec of the arguments each one accepts), and instead of writing prose, the model can output a structured request naming one of them.

The app, never the model, is the thing that actually runs it. The result gets stuffed back into the model's context, and it keeps going from there.

Using Claude's field names — OpenAI's function calling uses different keys, but the shape is identical:

// what the model outputs when it wants a tool
{
  "type": "tool_use",
  "id": "toolu_01A9q9",
  "name": "read_file",
  "input": { "path": "config.json" }
}

// what the app sends back
{
  "type": "tool_result",
  "tool_use_id": "toolu_01A9q9",
  "content": "{ \"port\": 4000 }"
}

Key takeaway: a "tool call" is just constrained text output. The model never leaves its own sandbox — the app on the other side does the leaving for it.

---

The loop, drawn out

Rose boxes and lines never leave the conversation. Gold ones cross into something real happening — a file read, an API call, a shell command.

---

Walking through it

1. The model reads its context. Before generating anything, it sees the conversation so far plus whatever tools are listed in the system prompt — each with a name, description, and schema. It never discovers tools on its own; they're handed to it, the same way my /ask route hands it five retrieved chunks.
2. It emits a tool_use block instead of prose. Still just next-token prediction, constrained to a shape the schema allows — not a separate reasoning step bolted on top.
3. The runtime executes it. The model can't touch a file system or an API. The app parses the request and performs the real action.
4. The result re-enters the context, and the loop repeats. The app appends the output as a tool_result. The model runs again, now able to see it, and either calls another tool or writes the final answer.

A single reply can cycle through this loop many times — read a file, search based on what it found, then edit it — before showing any text back. What looks like "planning" is this exact loop, iterated.

---

Why my chat skips all of this

Manual retrieval is simpler to reason about, and simpler was the right call for a corpus that fits in a handful of embeddings. If I ever wanted the chat to pull a specific post on demand, check something outside the static index, or hit a live source instead of a build-time snapshot, that's exactly the point where I'd expose retrieveTopK as a tool the model calls itself, instead of always running it up front.

---

Gotchas

• Picking a tool is prediction, not judgment. There's no separate reasoning module weighing options — the "choice" is shaped entirely by how well each tool's description matches the request. Vague or overlapping descriptions cause real misfires.
• The schema blocks malformed calls, not wrong ones. A request with a made-up file path still passes validation. Schemas check shape, not truth.
• There's no separate planner. Multi-step tool use is this exact loop, run repeatedly. The plan emerges turn by turn, not upfront.

Strip the branding off any coding agent, research bot, or support assistant, and this is the loop underneath. The product work is almost entirely in steps 1 and 3 — which tools you expose, how well you describe them, and what the runtime is allowed to touch.