AgentNotebook
Tutorials
Sofia Nieves9 min read4 views

toModelOutput is not called when your tool throws, and the raw error goes to the provider

On ai 7.0.114, whatever your tool execute function throws is serialised and sent to the model provider in a tool_result block marked is_error. The documented toModelOutput hook is skipped on that path. Measured first-party with a local capture server, no API key.

Flat printed schematic: a node feeds a diamond junction, one clean route passes through a hollow gate to the destination, and a second route carrying a single amber token bypasses the gate entirely and reaches the same destination.
Flat printed schematic: a node feeds a diamond junction, one clean route passes through a hollow gate to the destination, and a second route carrying a single amber token bypasses the gate entirely and reaches the same destination.
On this page

Vercel Anthropic Node.js Zod

Quick answer

As of September 2026, on ai 7.0.114 with @ai-sdk/anthropic 4.0.63, anything your tool execute function throws is turned into text and sent to the model provider inside the next request, in a tool_result block marked is_error: true. The toModelOutput hook, which is the documented way to control what the model sees, is not called on that path at all. A standard Error contributes its name and message; a thrown plain object is JSON.stringify-ed whole, every field included. The fix is not a hook: catch inside execute and return a value.

The reproduction, in about a minute

No API key and no network calls to a real provider. A local server stands in for the Anthropic Messages API, records every request body, and always answers with one tool call followed by a final text turn. This is the same local capture server technique we use whenever the question is what an SDK actually puts on the wire.

bash
mkdir ai-sdk-tool-errors && cd ai-sdk-tool-errors
npm init -y
npm install --save-exact ai@7.0.114 @ai-sdk/anthropic@4.0.63 zod@4.6.5

Versions are pinned because this article is about behaviour at a specific point in the tree. Node 24.8.0 here. The same provider package also decides request fields from a substring of your model id, which is a separate surface with a similar flavour.

javascript
// capture-server.js
// A local stand-in for the Anthropic Messages API.
// It records every request body to calls.json and always answers with
// one tool call, then a final text turn. No API key, no network, no cost.
const http = require('http');
const fs = require('fs');

function sse(res, data) {
  res.write('event: ' + data.type + '\n');
  res.write('data: ' + JSON.stringify(data) + '\n\n');
}

function alreadySentToolResult(payload) {
  const messages = (payload && payload.messages) || [];
  for (const message of messages) {
    const parts = Array.isArray(message.content) ? message.content : [];
    for (const part of parts) {
      if (part && part.type === 'tool_result') return true;
    }
  }
  return false;
}

const server = http.createServer(function (req, res) {
  let body = '';
  req.on('data', function (chunk) { body += chunk; });
  req.on('end', function () {
    let payload = null;
    try { payload = JSON.parse(body); } catch (err) { payload = body; }

    let calls = [];
    try { calls = JSON.parse(fs.readFileSync('calls.json', 'utf8')); } catch (err) { calls = []; }
    calls.push(payload);
    fs.writeFileSync('calls.json', JSON.stringify(calls, null, 2));

    const done = alreadySentToolResult(payload);

    if (!payload || !payload.stream) {
      const reply = done
        ? { id: 'm2', type: 'message', role: 'assistant', model: 'x',
            content: [{ type: 'text', text: 'Final answer.' }],
            stop_reason: 'end_turn', usage: { input_tokens: 20, output_tokens: 7 } }
        : { id: 'm1', type: 'message', role: 'assistant', model: 'x',
            content: [
              { type: 'text', text: 'Let me check.' },
              { type: 'tool_use', id: 'toolu_01', name: 'lookup', input: { city: 'Paris' } }
            ],
            stop_reason: 'tool_use', usage: { input_tokens: 10, output_tokens: 5 } };
      res.writeHead(200, { 'content-type': 'application/json' });
      res.end(JSON.stringify(reply));
      return;
    }

    res.writeHead(200, { 'content-type': 'text/event-stream', 'cache-control': 'no-cache' });
    sse(res, { type: 'message_start', message: { id: 'ms', type: 'message', role: 'assistant', model: 'x', content: [], stop_reason: null, usage: { input_tokens: 10, output_tokens: 0 } } });
    if (!done) {
      sse(res, { type: 'content_block_start', index: 0, content_block: { type: 'text', text: '' } });
      sse(res, { type: 'content_block_delta', index: 0, delta: { type: 'text_delta', text: 'Let me check.' } });
      sse(res, { type: 'content_block_stop', index: 0 });
      sse(res, { type: 'content_block_start', index: 1, content_block: { type: 'tool_use', id: 'toolu_01', name: 'lookup', input: {} } });
      sse(res, { type: 'content_block_delta', index: 1, delta: { type: 'input_json_delta', partial_json: '{"city":"Paris"}' } });
      sse(res, { type: 'content_block_stop', index: 1 });
      sse(res, { type: 'message_delta', delta: { stop_reason: 'tool_use' }, usage: { output_tokens: 5 } });
    } else {
      sse(res, { type: 'content_block_start', index: 0, content_block: { type: 'text', text: '' } });
      sse(res, { type: 'content_block_delta', index: 0, delta: { type: 'text_delta', text: 'Final answer.' } });
      sse(res, { type: 'content_block_stop', index: 0 });
      sse(res, { type: 'message_delta', delta: { stop_reason: 'end_turn' }, usage: { output_tokens: 7 } });
    }
    sse(res, { type: 'message_stop' });
    res.end();
  });
});

server.listen(8799, function () { console.log('capture server on 8799'); });

Three versions of the same tool: one that throws, one that throws and also defines toModelOutput, and one that catches its own failure.

javascript
// probe.mjs -- run as: node probe.mjs CASE
import { createAnthropic } from '@ai-sdk/anthropic';
import { generateText, streamText, tool, stepCountIs } from 'ai';
import { z } from 'zod';

const anthropic = createAnthropic({
  baseURL: 'http://127.0.0.1:8799/v1',
  apiKey: 'placeholder'
});

const DSN = 'connect ECONNREFUSED 10.0.3.14:5432';
const caseName = process.argv[2];

const base = {
  description: 'Look up the weather for a city',
  inputSchema: z.object({ city: z.string() })
};

const tools = {
  // 1. a tool that throws, nothing else
  plain: tool({
    ...base,
    execute: async function () { throw new Error(DSN); }
  }),
  // 2. the documented hook, hoping it will redact the error
  hook: tool({
    ...base,
    execute: async function () { throw new Error(DSN); },
    toModelOutput: function () {
      return { type: 'text', value: 'REDACTED' };
    }
  }),
  // 3. the fix: never let the throw escape execute
  caught: tool({
    ...base,
    execute: async function () {
      try {
        throw new Error(DSN);
      } catch (err) {
        return { ok: false, error: 'lookup_failed', retryable: true };
      }
    }
  })
};

const shared = {
  model: anthropic('claude-sonnet-4-5-20250929'),
  stopWhen: stepCountIs(5),
  prompt: 'What is the weather in Paris?'
};

if (caseName === 'stream') {
  const result = streamText({ ...shared, tools: { lookup: tools.plain } });
  for await (const part of result.textStream) { void part; }
  console.log('steps: ' + (await result.steps).length);
} else {
  const result = await generateText({ ...shared, tools: { lookup: tools[caseName] } });
  console.log('steps: ' + result.steps.length);
}

And a reader that prints the part of the second request that matters.

javascript
// reader.mjs -- prints the tool_result block from the second request
import fs from 'fs';

const calls = JSON.parse(fs.readFileSync('calls.json', 'utf8'));
if (calls[1] === undefined) {
  console.log('no second request was made');
} else {
  const messages = calls[1].messages;
  for (const message of messages) {
    const parts = Array.isArray(message.content) ? message.content : [];
    for (const part of parts) {
      if (part && part.type === 'tool_result') {
        console.log(JSON.stringify(part, null, 2));
      }
    }
  }
}

Start the server in one terminal, then in another run each case, deleting calls.json between runs.

What the wire actually shows

Case plain, a tool that simply throws:

text
steps: 2
{
  "type": "tool_result",
  "tool_use_id": "toolu_01",
  "content": "Error: connect ECONNREFUSED 10.0.3.14:5432",
  "is_error": true
}

The agent loop did not die. That part is intended and documented. What is easy to miss is the second half: the string connect ECONNREFUSED 10.0.3.14:5432 is now in a request body that goes to a third party. That is Node's own error text for a failed TCP connection, and it contains an internal address and port that nobody chose to publish.

Case hook, the same tool with toModelOutput added:

text
steps: 2
{
  "type": "tool_result",
  "tool_use_id": "toolu_01",
  "content": "Error: connect ECONNREFUSED 10.0.3.14:5432",
  "is_error": true
}

Byte for byte identical. The hook did not fire. If you add a counter inside toModelOutput it stays at zero on this path, and increments normally when the tool returns instead of throwing.

Case caught, the same failure handled inside execute:

text
steps: 2
{
  "type": "tool_result",
  "tool_use_id": "toolu_01",
  "content": "{\"ok\":false,\"error\":\"lookup_failed\",\"retryable\":true}"
}

No internal address, no is_error, and the model gets a structured signal it can actually reason about.

Why: one function decides, and it checks the error first

The whole behaviour lives in a single function in ai/dist/index.js. Reading it settles the question faster than any amount of experimenting. The shipped file is bundled, so the version below is lightly reformatted: the bundler's numeric suffixes are dropped and the compiled null check is written back as optional chaining. Nothing else is changed.

javascript
async function createToolModelOutput({ toolCallId, input, output, tool, errorMode }) {
  if (errorMode === "text") {
    return { type: "error-text", value: getErrorMessage(output) };
  } else if (errorMode === "json") {
    return { type: "error-json", value: toJSONValue(output) };
  }
  if (tool?.toModelOutput) {
    return await tool.toModelOutput({ toolCallId, input, output });
  }
  return typeof output === "string"
    ? { type: "text", value: output }
    : { type: "json", value: toJSONValue(output) };
}

The toModelOutput check sits below the two errorMode branches. The call site for generateText passes errorMode: output2.type === "tool-error" ? "text" : "none", so a thrown tool error always takes the first branch and returns before the hook is ever considered. This is not a bug report and I am not claiming it is unintended. It is a design where the escape hatch is only wired to the success path, and the consequence is not written down anywhere I could find. The source is public if you want to read the unbundled version in the vercel/ai repository.

The is_error: true flag is added one layer further out, by the Anthropic provider, which maps both error shapes onto the flag the Messages API tool-result block uses.

Exactly what leaves your process

getErrorMessage has exactly four branches, and between them they fully determine the leak surface:

javascript
function getErrorMessage(error) {
  if (error == null) {
    return "unknown error";
  }
  if (typeof error === "string") {
    return error;
  }
  if (error instanceof Error) {
    return error.toString();
  }
  return JSON.stringify(error);
}

Measured, throwing each of these from a tool:

Scroll to see more

What you throwWhat the provider receives
new Error('connect ECONNREFUSED 10.0.3.14:5432')Error: connect ECONNREFUSED 10.0.3.14:5432
an Error with name set to DatabaseErrorDatabaseError: timeout
an Error carrying a custom dsn propertyError: query failed
a bare stringthe string, verbatim
a plain object { code, dsn, port }the whole object, JSON.stringify-ed
null or undefinedunknown error

Two rows deserve attention, and they point in opposite directions.

The third row is the reassuring one. Error.prototype.toString returns only name and message, so extra properties hung on an error instance do not travel. I expected them to, and they do not. A stack trace does not travel either.

The fifth row is the dangerous one. Throw a plain object and every field is serialised. Object literals are a perfectly normal way to signal failure, and error objects assembled by hand often carry exactly the connection details you would never paste into a prompt. The general lesson is the same one that applies to what you hand a shell tool: the boundary is wherever your data stops being yours, and a tool body is on the wrong side of it.

streamText behaves the same way

Running the stream case against the same harness, with the server answering server-sent events instead of JSON:

text
steps: 2
{
  "type": "tool_result",
  "tool_use_id": "toolu_01",
  "content": "Error: connect ECONNREFUSED 10.0.3.14:5432",
  "is_error": true
}

Identical. The streaming path has its own call site, and it selects errorMode with the same ternary, so there is nothing here that only affects the non-streaming API.

The success path has its own surprises

Since one function converts every tool output, it is worth knowing what it does when nothing goes wrong. Non-string outputs go through toJSONValue, which is JSON.parse(JSON.stringify(value)) wrapped in two guards that turn undefined into null. That inherits every quirk of JSON.stringify, and a couple of them are sharp.

Here is the probe that produced the table. It walks a list of sample return values, runs each one through a real generateText call, and prints the exact content string the provider was handed.

javascript
// serialize.mjs -- what survives the trip from your tool to the model
import { createAnthropic } from '@ai-sdk/anthropic';
import { generateText, tool, stepCountIs } from 'ai';
import { z } from 'zod';
import fs from 'fs';

const anthropic = createAnthropic({
  baseURL: 'http://127.0.0.1:8799/v1',
  apiKey: 'placeholder'
});

const circular = { id: 1 };
circular.self = circular;

const samples = [
  ['plain object', function () { return { city: 'Paris', tempC: 21 }; }],
  ['array', function () { return [1, 2, 3]; }],
  ['string', function () { return 'plain text'; }],
  ['number', function () { return 42; }],
  ['undefined', function () { return undefined; }],
  ['Date', function () { return { at: new Date('2026-01-02T03:04:05Z') }; }],
  ['NaN and Infinity', function () { return { score: NaN, ratio: Infinity }; }],
  ['undefined field', function () { return { kept: 1, dropped: undefined }; }],
  ['function field', function () { return { kept: 1, cb: function () {} }; }],
  ['class instance', function () { class Row { constructor() { this.id = 7; } } return new Row(); }],
  ['Buffer', function () { return { blob: Buffer.from('hi') }; }],
  ['Map', function () { return { rows: new Map([['a', 1]]) }; }],
  ['Set', function () { return { tags: new Set(['x']) }; }],
  ['RegExp', function () { return { pattern: /^a.z$/i }; }],
  ['BigInt', function () { return { total: 9007199254740993n }; }],
  ['circular', function () { return circular; }]
];

for (const [label, make] of samples) {
  fs.writeFileSync('calls.json', '[]');
  const probe = tool({
    description: 'probe',
    inputSchema: z.object({ city: z.string() }),
    execute: async function () { return make(); }
  });
  let shown;
  try {
    await generateText({
      model: anthropic('claude-sonnet-4-5-20250929'),
      tools: { lookup: probe },
      stopWhen: stepCountIs(5),
      prompt: 'go'
    });
    const calls = JSON.parse(fs.readFileSync('calls.json', 'utf8'));
    shown = 'never sent';
    if (calls[1] !== undefined) {
      for (const message of calls[1].messages) {
        const parts = Array.isArray(message.content) ? message.content : [];
        for (const part of parts) {
          if (part && part.type === 'tool_result') shown = JSON.stringify(part.content);
        }
      }
    }
  } catch (err) {
    shown = 'THREW ' + err.name + ': ' + String(err.message).split('\n')[0];
  }
  console.log(label.padEnd(18) + shown);
}
text
plain object      "{\"city\":\"Paris\",\"tempC\":21}"
array             "[1,2,3]"
string            "plain text"
number            "42"
undefined         "null"
Date              "{\"at\":\"2026-01-02T03:04:05.000Z\"}"
NaN and Infinity  "{\"score\":null,\"ratio\":null}"
undefined field   "{\"kept\":1}"
function field    "{\"kept\":1}"
class instance    "{\"id\":7}"
Buffer            "{\"blob\":{\"type\":\"Buffer\",\"data\":[104,105]}}"
Map               "{\"rows\":{}}"
Set               "{\"tags\":{}}"
RegExp            "{\"pattern\":{}}"
BigInt            THREW TypeError: Do not know how to serialize a BigInt
circular          THREW TypeError: Converting circular structure to JSON

Three groups, with three different failure styles:

  • Silent emptying. A Map, a Set and a RegExp all arrive as {}. No warning, no error, no missing key. The model is handed an empty object where your data was, and then reasons confidently about nothing. If a tool returns counts in a Map, this is a correctness bug that never raises.
  • Loud crash. A BigInt or a circular reference throws a TypeError out of generateText and no second request is made at all. Adjacent JavaScript values, opposite outcomes.
  • Quiet flattening. NaN and Infinity become null, undefined fields and function fields vanish, and a class instance is reduced to its own enumerable properties.

The undefined row is worth one more sentence: a tool that returns nothing tells the model the string null, which reads as a definite answer rather than an absence.

What actually works

Catch inside execute and return a value. This is the only approach here that is both documented behaviour and under your control. You decide what the model sees, you keep the agent loop alive, and you can give the model something structured enough to retry against:

javascript
execute: async function ({ city }) {
  try {
    return await lookupWeather(city);
  } catch (err) {
    log.error({ err, city }, 'weather lookup failed');
    return { ok: false, error: 'lookup_failed', retryable: true };
  }
}

Note what this does with the real error: it goes to your logs, where it belongs, and a short code goes to the model. You are not throwing information away, you are routing it.

A caveat about the lifecycle callback. onToolExecutionEnd receives the tool output object, and if you mutate toolOutput.error in place, the mutated value is what gets serialised. I measured this and it reproduces on both the error and the success paths. I am recording it rather than recommending it. The docs describe these callbacks as observability, giving you visibility into tool execution, and say that "Errors thrown inside these callbacks are silently caught and do not break the generation flow." Nothing promises that mutating the argument is load bearing, so treat it as an implementation detail that can change in a patch release rather than as a redaction layer.

What does not work: toModelOutput, as shown above. It is genuinely useful for shaping successful results, which is what the reference describes, and it is simply not on the error path.

What the docs do and do not say

The vendor page is thorough about the intent. It states plainly that "When tool execution fails (errors thrown by your tool's execute function), the AI SDK adds them as tool-error content parts to enable automated LLM roundtrips in multi-step scenarios." That is the round trip working as designed, and it is a reasonable default: an agent that can see why a tool failed can often recover.

What the same 55,000-character page does not contain, measured: zero occurrences of errorMode, zero of is_error, and exactly two of toModelOutput, both inside a section about returning screenshots as multi-modal content. The interaction between the two is not described, and the phrase "adds them as tool-error content parts" does not obviously tell a reader that the raw message string is about to cross a network boundary.

Two things I expected and did not find

Recording the misses, because both would have made for a more dramatic and less accurate article.

Custom error properties do not leak. My first hypothesis was that a rich error object attached to an Error instance would be serialised wholesale. It is not. error.toString() means name and message only, so the dsn property I hung on a test error never appeared on the wire. The leak is real but narrower than I assumed, and it is much worse for thrown object literals than for thrown Error instances.

This is not an Anthropic-provider quirk. I expected to find the behaviour in @ai-sdk/anthropic, next to the is_error mapping. It is not there. The decision is made in the core ai package before any provider sees it, and the provider only translates the resulting shape. Any provider package built on the same core will behave the same way, which is why the fix belongs in your tool and not in your provider configuration.

The short version

If a tool can fail, decide what its failure looks like. Wrap the body in try/catch, log the real error locally, and return a small structured object. Do not reach for toModelOutput to clean up after a throw, because on that path it is never called, and do not assume a Map survived the trip.

#ai-sdk#tools#error-handling#anthropic#typescript#agents
Back to tutorials
Share
Sofia Nieves

Written by

Sofia Nieves

Sofia works on agent evaluation and reliability. She writes about measuring LLM systems before and after they reach production.

Frequently asked questions

Does toModelOutput run when my tool throws?

No. Measured on ai 7.0.114 and @ai-sdk/anthropic 4.0.63, toModelOutput is not called at all on the error path. The conversion function checks the errorMode argument first and returns before it reaches the hook, so a counter placed inside toModelOutput stays at zero when the tool throws and increments normally when it returns.

What exactly does the provider receive when my tool throws?

A tool_result block with is_error set to true, whose content is the output of getErrorMessage. A standard Error contributes name and message via Error.prototype.toString. A thrown string is passed through verbatim. A thrown plain object is JSON.stringify-ed in full, every field included. Throwing null or undefined produces the literal text unknown error.

Do custom properties on an Error object leak?

No. Error.prototype.toString returns only name and message, so a dsn or config property attached to an Error instance does not travel, and neither does the stack. This is the one place the exposure is narrower than you might expect. Thrown plain objects are the dangerous case, because those are serialised whole.

Is streamText affected too, or only generateText?

Both. The streaming path has its own call site but selects errorMode with the same ternary, and a measured run against a server-sent-events harness produces a byte-identical tool_result block. The decision is made in the core ai package before any provider sees it, so provider packages other than Anthropic will behave the same way.

What is the correct way to control what the model sees after a failure?

Catch inside execute and return a value. Log the real error locally and return a small structured object such as an ok flag, a short error code and a retryable flag. That keeps the agent loop alive, keeps internal detail inside your process, and gives the model something it can actually act on.

Can I redact the error from onToolExecutionEnd instead?

Mutating toolOutput.error inside that callback does change what gets serialised, and it reproduces on both the error and the success paths. It is not a documented contract though. The docs describe those callbacks as observability, so treat this as an implementation detail that can change in a patch release rather than as a redaction layer.

Why did my tool result arrive at the model as an empty object?

Non-string tool outputs go through JSON.parse(JSON.stringify(value)). A Map, a Set and a RegExp all serialise to {} with no warning, so the model receives an empty object where your data was. A BigInt or a circular reference is worse in one sense and better in another: it throws a TypeError out of generateText, so at least it fails loudly.

Related tutorials

Tutorials

Argument injection: your AI agent's allowlisted shell tool is not safe (Node)

You gave your agent a shell tool, used execFile so there is no shell, and allowlisted the programs. It is still not safe. If the agent controls the argument array, tar's own --checkpoint-action=exec runs any command with no metacharacters. Here is the reproduction and the per-binary argument policy that stops it, measured on Node 24.8.0.

8 min read30