Agentic Workflow Routing: Build an LLM Router in Python (2026)

Quick Answer

Routing is an agentic workflow pattern where a lightweight classifier reads an incoming request, labels it, and sends it to a specialized handler built for that label. As of July 2026 it is the second of the five patterns in Anthropic's "Building Effective Agents" taxonomy, sitting one step up from prompt chaining. Routing buys you two things: cleaner, more focused prompts per category, and the option to send easy work to a cheap model and hard work to a strong one. This tutorial builds a runnable router in Python with the Claude API, adds a deterministic fallback so an unknown label can never crash the pipeline, and shows the cost-tiered variant that routes by difficulty.

What routing is

Anthropic defines the pattern precisely in Building Effective Agents: "Routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts."

The shape is a fork, not a chain. Prompt chaining runs a fixed sequence of steps for every input. Routing runs a classifier once, then picks exactly one downstream branch. That single decision is the whole pattern. Everything else is engineering the decision so it stays reliable.

                        +-------------------+
   incoming request --> |  router / classify | --(label)-->  branch
                        +-------------------+
                                                    |
                    +-------------------------------+-------------------------------+
                    v                               v                               v
            +---------------+             +-----------------+             +-----------------+
            | billing agent |             | technical agent |             |  general agent  |
            +---------------+             +-----------------+             +-----------------+

When to reach for it (and when not)

Use routing when your inputs fall into distinct categories that are genuinely handled differently, and when the category can be decided reliably. Two cases from Anthropic's writeup make it concrete: directing customer service queries (general questions, refunds, technical support) into separate tools, and routing straightforward questions to a smaller model while sending difficult ones to a stronger model for a better cost/quality trade.

Skip it when the categories bleed into each other. If the classifier is wrong 30% of the time, you have not separated concerns, you have added a failure mode. When boundaries are fuzzy, a single well-written prompt often beats a shaky router. A reliable classifier is the load-bearing assumption of the entire pattern.

Assumptions

• Python 3.10 or newer.
• The official anthropic SDK: pip install anthropic (1.x line, current as of July 2026).
• An API key in the ANTHROPIC_API_KEY environment variable.
• Model names below are the current 2026 aliases: claude-haiku-4-5 for cheap, fast classification and claude-sonnet-5 for the hard branch. Check the live Anthropic API reference for the exact model strings and the pricing page for current per-token rates before you ship.

the classifier

The router is one small call. Constrain it hard: give it a closed set of labels and tell it to answer with a single word. A tiny max_tokens keeps it cheap and makes a runaway response impossible.

import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

ROUTES = ["billing", "technical", "general"]

def classify(request: str) -> str:
    prompt = (
        "Classify the support request into exactly one category.\n"
        f"Categories: {', '.join(ROUTES)}.\n"
        "Reply with ONLY the category word, nothing else.\n\n"
        f"Request: {request}"
    )
    resp = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=5,
        messages=[{"role": "user", "content": prompt}],
    )
    label = resp.content[0].text.strip().lower()
    # deterministic fallback: an unexpected label can never crash the pipeline
    return label if label in ROUTES else "general"

The last line is the part people skip. A model can return Billing. or technical support or an apology. Validating against the closed ROUTES set and defaulting to general means a bad classification degrades quietly instead of throwing a KeyError three lines later.

the specialized handlers

Each branch gets its own system prompt. This is the payoff Anthropic points at: the billing prompt never has to hedge about refunds it will never see, so it can be sharper.

HANDLERS = {
    "billing": "You are a billing specialist. Be precise about charges, refunds, and invoices.",
    "technical": "You are a senior support engineer. Give exact, reproducible troubleshooting steps.",
    "general": "You are a friendly support agent. Answer clearly and offer a next step.",
}

def handle(request: str, route: str) -> str:
    resp = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=600,
        system=HANDLERS[route],
        messages=[{"role": "user", "content": request}],
    )
    return resp.content[0].text

def route_request(request: str) -> str:
    route = classify(request)
    return handle(request, route)

if __name__ == "__main__":
    print(route_request("I was charged twice for my October invoice."))

Run it. The classifier sees a charge complaint, returns billing, and the billing system prompt answers. Swap in a stack trace and it lands on technical. One decision, one branch.

route by difficulty to cut cost

The second Anthropic example is a pure economics play: classify difficulty, then pick the model. Send the easy 80% to Haiku and reserve Sonnet for the hard 20%.

def classify_difficulty(request: str) -> str:
    prompt = (
        "Is this request simple or hard to answer well?\n"
        "Reply with ONLY 'simple' or 'hard'.\n\n"
        f"Request: {request}"
    )
    resp = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=5,
        messages=[{"role": "user", "content": prompt}],
    )
    label = resp.content[0].text.strip().lower()
    return label if label in ("simple", "hard") else "hard"  # fail toward quality

def answer(request: str) -> str:
    model = "claude-sonnet-5" if classify_difficulty(request) == "hard" else "claude-haiku-4-5"
    resp = client.messages.create(
        model=model,
        max_tokens=600,
        messages=[{"role": "user", "content": request}],
    )
    return resp.content[0].text

Note the fallback direction flipped: an ambiguous difficulty defaults to hard, because paying for the stronger model is cheaper than shipping a wrong answer. Pick your fallback to fail toward whichever error you can least afford. The same idea works across providers; if you route between vendors, OpenAI's API docs cover the equivalent call shape.

Making the decision reliable

The router is a classifier, so treat it like one.

• Closed label set, always validated. Never index a dict with raw model output. Check membership, then fall back.
• Keep the classifier cheap. A 5-token Haiku call is small next to the handler call, so the router adds latency and cost you barely notice.
• Log the label. When an answer is wrong, you need to know whether the router misfired or the handler did. They are separate bugs with separate fixes.
• Consider a non-LLM router. If a regex or a keyword match classifies reliably, use it. The pattern is about specialized branches, not about spending a token on every decision.

Routing is often the front door to the larger patterns. Once branches start calling each other or looping, you are moving toward orchestrator-workers and evaluator-optimizer, the two heavier patterns in the same taxonomy. Start here, and grow into those only when a single fork stops being enough. For the full map, see our agentic workflow patterns in Python overview, and for the pattern one rung down, the prompt chaining tutorial.

Limitations and open questions

• The classifier is a single point of failure. Every downstream answer inherits its accuracy. Measure classification accuracy on real traffic before you trust the branches.
• Two calls, two latencies. Routing adds a classification round trip. For latency-critical paths, a non-LLM classifier or a cached decision may be worth it.
• Category drift. Real inbound traffic mutates. A route set that fit in January may need a fourth branch by June; budget for periodic review of the label distribution.
• Open question: when does routing beat a single strong prompt? For narrow, high-volume categories the separation clearly helps. For long-tail, overlapping requests the answer is empirical. Log both and compare.