Evaluator-Optimizer: Build a Self-Correcting LLM Agent in Python (2026)
The evaluator-optimizer workflow puts a rubric-driven critic in the loop so a generator revises until it passes. A runnable Python build with real guardrails.
Tag
14 posts tagged.
The evaluator-optimizer workflow puts a rubric-driven critic in the loop so a generator revises until it passes. A runnable Python build with real guardrails.
Run independent LLM calls at the same time in Python with asyncio.gather, cap them with a Semaphore, and survive 429 rate limits with backoff. Runnable code.
Build the orchestrator-workers agent pattern in Python with Claude: a planner that decides subtasks at runtime, parallel workers, and a synthesizer. Runnable code, July 2026.
A code-first routing tutorial: build a runnable LLM router in Python with the Claude API, add a deterministic fallback, and route by difficulty to cut cost.
A code-first prompt chaining tutorial: build a gated Python pipeline with the Claude API, add pure-Python and LLM gates, and fail fast before wasting tokens.
You cannot unit-test an agent like a pure function. Build a two-layer pytest harness: deterministic tool-call assertions plus an LLM-as-judge grader, a frozen eval dataset, and a CI gate. Runnable Python, no eval framework required.
Five reusable agentic workflow patterns, built from scratch in Python with runnable code and a start-simple rule.
The five parts every AI agent has, a runnable Python example in about 40 lines, and an honest guide to the guardrails and framework choices most tutorials skip.
A runnable 2026 tutorial: build short-term and long-term memory for a Claude agent in TypeScript with message buffers, rolling summaries, and a durable fact store. No vector database required.
A runnable Python quickstart: install the Claude Agent SDK, stream a run with query(), then give the agent your own tools with the @tool decorator and ClaudeSDKClient. Builds a weather agent that chains two tools.
A runnable 2026 quickstart: install the Claude Agent SDK, wire a custom tool with tool() and createSdkMcpServer(), and let the agent loop call it for you in about 40 lines of TypeScript.
Should you use the Claude Agent SDK or hand-roll the agent loop with the Anthropic Client SDK? An honest 2026 comparison with runnable TypeScript for both and a decision table.