The agent desk: routing, worker agents and memory

Part 2 described what the model is allowed to output: a typed strategy document. This part covers how a message becomes that document, and how we kept the cost of doing so from sinking the product.

Two numbers frame the cost story. A request that went to the full reasoning model cost about $0.10 once you counted planning, tool calls and the explanation. A distilled router in front of it, with a semantic cache, avoided more than 90% of full LLM calls and brought the average to about $0.01 a query. This part covers:

• The intent classifier, and the distilled router with its semantic cache
• One orchestrator and five stateless worker agents, and how they map to the four jobs on the public site
• The staged workflow and the clarifying loop
• Memory: working memory and observational memory
• Retrieval: hybrid keyword and dense search
• Explanations, and the guardrails around them

Background: the cost problem#

Traders are chatty. A session is a mix of “price of RELIANCE?”, “show me that on a 15-minute chart”, “what’s the PCR on Bank Nifty”, “same thing but with a 1.5% stop”, and occasionally “build me a strategy that…”. If every message goes to the reasoning model with the full system prompt and tool list, you pay for deep reasoning on greetings and lookups.

The fix isn’t a cheaper reasoning model. It’s not calling one when you don’t need to. That needs a component that’s fast, cheap and right about which messages need reasoning.

Intent classification and the router#

The intent classifier#

The backend has an intent classifier that runs before the workflow. It tries quick pattern matches first, and when those don’t settle it makes an LLM call that returns a domain, an action, a short reasoning and the parameters it extracted:

• Domain: STRATEGY (backtests, strategy creation, optimization), SCREENER (scanning and filtering), CONVERSATION (explanations and help) or SYSTEM (cancel, reset, status).
• Action: NEW (start fresh), CONTINUE (confirm and proceed), CLARIFY (answering a pending question), MODIFY (change parameters without starting over) or SWITCH (drop this for another domain).

The classifier also reads the session state: the current domain, the stage and any pending questions. That’s what makes a bare “2%” mean “the stop-loss you just asked about,” not a new request.

The router and the semantic cache#

In front of the reasoning model sits a cheaper tier, “the triage nurse”: a lightweight Llama-3 8B, distilled to do one job, classification, together with a semantic cache. A question like “price of RELIANCE” that was answered seconds ago for another user is served without calling a large model at all. The reasoning agents run on GPT-4.1, and memory compression runs on Gemini 2.5 Flash, so the router decides which messages ever reach the expensive tier. It runs on modest CPU hardware, with classification in under 50 ms.

graph TD
    M["User message\n+ session state"] --> C{"Semantic cache\nsame question,\nsame state?"}
    C -->|hit, cacheable| H["Return cached\nresult"]
    C -->|miss| R["Router model\ndistilled Llama-3 8B"]
    R -->|lookup| T["Direct tool call"]
    R -->|edit| E["Patch last strategy\n+ recompile"]
    R -->|strategy, screen| L["Reasoning model\n+ workflow"]

    class C,R amber
    class H,T green
    class E violet
    class L red

What the cache may return#

A similarity match alone isn’t enough, because in markets the same question has a different answer every minute. The cache is in Redis, and a hit needs cosine similarity above 0.92 on the embedded message. It also needs a state key next to the similarity: for quotes, the bar boundary; for screens, the last closed bar; for backtests, the sir_hash and the data version. A cached strategy document that is reused still goes back through the compiler, and anything live never comes from the cache. The 0.92 threshold was tuned against reviewed hits.

The asymmetric cost of errors shaped the decision rule more than the model. Sending a lookup to the reasoning model wastes a few cents. Sending a strategy request down the lookup path produces a wrong answer. So the router escalates whenever it’s unsure.

One orchestrator, five workers#

The backend has one user-facing orchestrator agent. Its only tool is a supervisor tool that moves the conversation through stages and calls five worker agents: strategy, risk, backtest, performance and market scanner. The workers are stateless single-turn processors. They don’t use memory, and their outputs are normalized before the next stage uses them.

An earlier design, described in my first post on āagman, listed five specialists: a market scanner, a strategy agent, a risk analyst, an execution agent and a performance analyst. The execution agent became a backtest worker, because the model never places orders: execution belongs to the execution service (Part 7).

The four jobs on the site#

The public site describes four agents: charting, research, backtesting and deploy. Those are the four user-facing jobs, grouped by what a trader wants to do. Each is served by the same orchestrator and a different mix of workers and tools. The following table maps the two views:

No agent can place an order. The orchestrator’s only tool is the supervisor tool, the workers have no broker access, and the order path starts only in the execution service. That’s the property from Part 1: the model’s reach ends at the typed document.

A staged workflow, not free loops#

The supervisor tool keeps a session with a stage and advances it: strategy, then risk, then backtest, then performance, then complete, with a separate screening stage for scans. A CONTINUE moves to the next stage only when nothing is pending; a MODIFY goes back and changes parameters; a SWITCH changes domain.

graph LR
    S["Strategy\nworker"] --> Q{"Questions\npending?"}
    Q -->|yes| A["Ask the user"]
    A --> S
    Q -->|no| R["Risk\nworker"]
    R --> B["Backtest\nworker"]
    B --> P["Performance\nworker"]
    P --> D["Explain results\n+ caveats"]

    class Q amber
    class A violet
    class S,R,B,P blue
    class D green

Staged steps meant a malformed intermediate output failed at the step that produced it, and progress could be shown per stage.

The clarifying loop#

A strategy stays in DRAFT until its assumptions are confirmed and its clarifications are cleared (Part 7 lists the full readiness gate). Each document carries an explainability section: the assumptions the model made, each flagged if it needs confirmation, and a list of clarifications still needed, each tied to a field path. That’s how āagman asks for missing pieces instead of guessing, which is the behaviour the public guide promises.

Users write in English and in Indian languages; the assistant prompts for the earlier product explicitly covered Hindi, Gujarati, Tamil, Kannada and Telugu. The strategy document is always in the same canonical form, so nothing downstream needs to know the input language.

Memory#

The orchestrator has two layers of memory, built on Mastra:

• Working memory, scoped to the user: a scratchpad for preferences such as markets, indicators and risk tolerance, which persists across conversations.
• Observational memory, scoped to a conversation thread: it compresses older messages into dense observations and reflections, so a long thread keeps its context. The documentation puts the compression at 5 to 40 times and runs it on a small, cheap model.

Both are stored in Postgres. Retrieval over longer-term memory is hybrid, described next. The performance worker also writes “lessons learned” entries back after a trade, and the strategy worker retrieves things like “past strategies this user rejected” before it drafts.

Retrieval#

Memory retrieval is hybrid: BM25 keyword search runs next to dense search on pgvector with an HNSW index. Financial text needs both. Tickers, scheme codes and “Q2 FY26” are tokens that dense embeddings blur, so the keyword side catches them, while the dense side catches paraphrase. Filters are applied before ranking.

The vectors stay in Postgres rather than a separate vector database, for transactional consistency with the rows the vectors describe.

Explanations#

The last step of the workflow is an explanation: what was run, what assumptions were filled in, and what the results mean. The explanation is generated from structured inputs (the compiled plan, the metrics, the trade list), not from the conversation, so it describes what actually ran. Two rules live in code, not in the prompt: numbers in the text must come from the data, and required caveats are appended by code so the model can’t drop them.

What I’d do differently#

Route on cost, not only on intent. The router predicted intent and we mapped intent to a path. A better router predicts the cheapest path that will produce a correct result, trained on outcomes (did the user accept the answer, edit it, or rephrase?).

Measure cache correctness, not only hit rate. Hit rate is simple to put on a dashboard. A wrong cache hit is silent. I’d sample hits for review from day one.

Conclusion#

The agent desk is where the model earns its place: reading messy, multilingual intent and producing a precise document. Most of the engineering went into keeping it away from everything else: a classifier so it’s used on purpose, a staged workflow so its steps are typed, and workers with narrow jobs. The next part covers the engine that runs that document: the backtester.

I built āagman as Founding Software Engineer at KotiLabs, July 2025 to April 2026.