Tag

#testing

5 posts tagged.

Tutorials

Your MCP stdio server keeps running after the client dies (Node)

An MCP stdio server that holds a timer or a pool open does not exit when its client dies. Measured on SDK 1.30.0 and Node 24.8.0: it is reparented to init and keeps running. StdioServerTransport registers no stdin end-of-file listener. Here is the reproduction and the four-line fix.

9 min read139
Tutorials

How to sandbox an AI agent's file tools in Node.js

If your agent has a read_file tool, you probably guard it with path.resolve plus startsWith. It blocks the obvious traversal and is defeated by a symlink and by a sibling directory sharing the prefix. Both reproduced on Node 24.8.0, with the guard that passes all nine cases.

8 min read231
Eval

Agent eval methodology: 5 metrics that actually catch regressions

Agents fail quietly: a prompt tweak that fixes one task often breaks three others, and manual spot-checks never re-test what used to work. The fix is a frozen eval set scored on every change. This tutorial builds that harness and tracks five metrics that actually catch regressions, task success rate, tool-call accuracy, step efficiency, cost per task, and a safety/guardrail rate. You will assemble an eval set, write a runner that scores each metric, and turn the before/after diff into a regression gate so a change only ships when the numbers hold or improve.

6 min read617