Routing

The Core Idea

No single prompt can be excellent at everything. The wider the variety of requests you push through one do-everything prompt, the more average it becomes at each — every instruction you add for one kind of request is noise for the rest. The routing pattern escapes this by looking at each input before attempting it, and steering it onto whichever specialised path is best equipped to handle it. Where the input comes from doesn't matter — a user typing a request and an upstream planning step handing work down are routed exactly the same way.

Airports solve the identical problem with floor signage: one stream of arriving passengers splits into transfers, passport control, and baggage enquiries — each lane staffed and equipped for exactly one job, and nobody queues behind a problem that isn't theirs.

Why Route?

The Two Stages: Classification and Dispatch

Every router, however sophisticated, does exactly two things in sequence.

Stage 1 — Classification. Work out what the input is before anyone attempts it: what is being asked for, which category of work it belongs to, what the user is really trying to achieve — and sometimes how demanding answering it will be. In a travel assistant this is the moment of recognising "this is a flight-change request", before anyone tries to answer it.

Stage 2 — Dispatch. Act on the label: hand the input, together with the label that explains it, to the destination built for that kind of work — which might be a specialist agent, an entire prompt chain, or nothing more exotic than an ordinary function. This is the branching logic itself: label "flights" takes the flights path to the flights agent, label "hotels" takes the hotels path. The flight-change request lands with the flights desk, not whoever happened to pick up the phone.

The two stages are two halves of one design: every label the classifier can emit needs a route built to handle it. Design the two lists as a pair — the labels the classifier may answer with, and the specialists who handle them — so that each route is good at exactly one label's kind of work, and the hand-off practically chooses itself.

Ways to Classify

Choosing between them comes down to three questions: how messy is the input? (free-form language pushes you towards an LLM; structured fields can stay rule-based), how expensive is a misroute? (the costlier the mistake, the smarter the classifier needs to be), and what does each decision cost? (latency and money, multiplied by every input you'll ever receive). Many production routers layer the methods — rules catch the obvious cases instantly, and only ambiguous inputs pay for the LLM call.

Routing in Code

The whole pattern is three short pieces: the classifier (stage 1) that reads a request and returns a label, the specialists it can route to, and the dispatch table (stage 2) that connects a label to its specialist. None of them is more than a handful of lines.

Stage 1 is a deliberately tiny LLM call — one category in, one word out. (chat() below is the small helper used across this series: it sends a prompt to the LLM and returns the text reply.)

def classify_request(request):
    """One cheap, deterministic LLM call that returns a label."""
    label = chat(
        user_prompt=f"Classify this traveller request into exactly one category.\n\nRequest: {request}",
        system_prompt=(
            "You are a triage classifier for a travel assistant. Reply with "
            "exactly one word: flights, hotels, or activities."
        ),
        max_tokens=5,       # tiny output — fast and cheap
        temperature=0.0,    # classification should be deterministic
    )
    return label.strip().lower().rstrip(".")

Next, the destinations. Each specialist is an ordinary Python function with one focused persona — the flights specialist knows only flights, and so on. They are plain functions, not part of the routing machinery; the router's only job is to pick the right one. Here is the flights specialist; the rest differ only in their system prompt:

def flights_agent(request):
    """Specialist: bookings, changes, and cancellations."""
    return chat(
        user_prompt=request,
        system_prompt=(
            "You are a flight specialist. Handle bookings, date changes, "
            "cancellations, and baggage questions. Be precise about dates, "
            "fees, and the exact next steps the traveller should take."
        ),
        max_tokens=300,
    )

# hotels_agent, activities_agent, and general_agent follow the same shape —
# same signature, different persona. general_agent is the fallback route.

Stage 2 — dispatch — is the lookup that connects the two. The dictionary is the route table: hand it a label, get back the specialist that handles it. run_router then ties the pattern together in three lines — classify, look up, call:

ROUTES = {
    "flights": flights_agent,
    "hotels": hotels_agent,
    "activities": activities_agent,
}

def run_router(request):
    label = classify_request(request)
    specialist = ROUTES.get(label, general_agent)  # unknown label → safe fallback
    print(f"[Router] {label} → {specialist.__name__}")
    return specialist(request)

That one lookup line does two useful things at once. ROUTES.get(label, general_agent) reads as: "find the specialist for label; if there is no such label, use general_agent instead." Since the label comes from an LLM — which can ignore "reply with one word" and return something off-list — those two things are exactly what you need: first it checks the label is one we actually handle, and second it has a safe fallback ready when it isn't, so an unrecognised label quietly lands on the general agent instead of crashing the workflow. (Plain ROUTES[label] would throw an error on any unexpected label.)

Printing the routing decision is the seed of observability: when a request gets a strange answer, the first question is "where was it routed, and why?"

Send three requests through and each takes its own path:

Request: Can I move my Rome flight from Tuesday to Thursday?
[Router] flights → flights_agent

Request: Does the hotel in Lisbon include breakfast?
[Router] hotels → hotels_agent

Request: What should we do in Barcelona with two kids for a weekend?
[Router] activities → activities_agent

Growing the Router

The minimal router is genuinely useful as-is, but three upgrades appear in almost every production version.

1. Self-describing specialists

In the minimal router the categories are written out by hand in two places that must agree: the classifier's system prompt (which lists the labels) and the route table (which maps each label to a specialist). Adding a specialist means editing both. The fix: keep the specialists in one list — each entry a name, a one-line description, and the function that does the work — and build the system prompt from that list each time the router runs.

SPECIALISTS = [
    {"name": "flights", "description": "bookings, date changes, cancellations, baggage", "agent": flights_agent},
    {"name": "hotels",  "description": "rooms, amenities, check-in times, reservations",  "agent": hotels_agent},
    # adding a specialist = adding one entry — the routing logic never changes
]

def build_router_prompt(specialists):
    # one line per specialist, e.g. "- flights: bookings, date changes, ..."
    lines = [f"- {s['name']}: {s['description']}" for s in specialists]
    text = "\n".join(lines)  # stack the lines into one block of text
    return (
        "You are a triage classifier for a travel assistant.\n"
        "Pick the best specialist for the request from this list:\n"
        f"{text}\n"
        "Reply with exactly one name from the list. No punctuation, no explanation."
    )

# each time the router runs, it rebuilds its system prompt from the list:
system_prompt = build_router_prompt(SPECIALISTS)

The function does nothing clever — it just glues the list into the classifier's system prompt. For the two-entry list above, system_prompt is exactly what the LLM receives:

You are a triage classifier for a travel assistant.
Pick the best specialist for the request from this list:
- flights: bookings, date changes, cancellations, baggage
- hotels: rooms, amenities, check-in times, reservations
Reply with exactly one name from the list. No punctuation, no explanation.

lines = []
for s in specialists:                              # s is one specialist entry at a time
    lines.append(f"- {s['name']}: {s['description']}")

2. Short lists, cascaded routers

Every option you add makes the router's decision harder: the whole list sits in the prompt, the distinctions between specialists get finer, and classification accuracy drifts down. Keep each router's list short. When you genuinely need more destinations, cascade: a route's destination can itself be another router. The top one makes a coarse choice between a few areas; each area's router makes a finer one. Every individual decision stays easy, and the tree handles the scale.

3. Composite routes

Some requests don't fit a single hand-off. Ask "roughly what would a week in Lisbon cost for a family of four?" and no single specialist can answer — fares live with the flights desk, room rates with hotels, and someone still has to put the numbers together. Rather than reviving the generalist with one prompt that covers everything, make the route itself a mini-chain: when the classifier says "budget", the router gathers the prerequisite research from the existing specialists, then hands it all to a budget planner whose only job is synthesis.

The budget is only as good as the research behind it — weak gathering means a weak budget. The budget route normally gathers that research itself by calling the flights and hotels specialists, but if the code that called the router already has it — fetched earlier for some other reason — it can hand the research in through an optional context argument, and the router uses it directly instead of fetching it again.

Lesson Recap