MCP vs CLI vs Code — pick none of the above, and combine them.

Polu opened with the deceptively boring question of how an agent calls a tool. The three default answers each break on a specific axis: code-or-CLI loses governance, MCP loses composability and floods the context window, and naked shell loses an audit trail. The comparison was sharp enough to print directly:

What this means in practice

The synthesis is a thin CLI binary that speaks MCP under the hood. The agent gets POSIX-style composability (the thing humans love about shells) while the platform keeps MCP's auth, enumeration, and audit-trail properties. Lazy schema loading is the punchline: you only pay for the tool descriptions you actually use, which dissolves the 55k-token bloat of an unfiltered GitHub MCP server.

The three properties that survive into the merged design are worth memorising as a checklist for any tool-calling design you do: Auth Discovery, Action Enumeration, Typed Audit Trails. If your tool plumbing doesn't have all three, you've built either a prototype or a liability.

From a sync loop to an unstoppable workflow.

A while-loop that calls a model, runs tools, and feeds results back is the textbook agent. It's also fragile in three specific ways: loops last hours, deploys resist completing, and any approval pause is fatal. The frame Polu used — "problem / constraint / solution" — is a clean diagnostic structure for any team facing the same wall.

The code, side by side

function agentLoop() {
  const state = [userMsg];
  for (;;) {
    // 🔥 deploy = lost state
    const r = await callModel(state);
    await runTools(r);
  }
}

workflow agentLoop() {
  for (;;) {
    // each step persisted to DB ✓
    await Activity.callAgent(step);
    await Activity.executeTool(step);
  }
}

The diff looks cosmetic — function becomes workflow, callModel becomes Activity.callAgent. The semantics are completely different. Every await in the durable version is a checkpoint that survives process death, deploys, network blips, and human approvals.

The universal tool layer — give the agent a Linux box, not bash.

The Universal Tool Layer is where Polu's design becomes opinionated. Instead of mapping every external system to a custom tool, Dust gives the agent one tool: a fresh Linux x86_64 sandbox per run, with everything pre-installed and a typed CLI on top. The agent issues bash commands; the sandbox enforces governance.

The architectural sleight-of-hand here is that the sandbox is per-run and isolated — it doesn't survive past the conversation. Persistence happens at the durable-workflow layer (§1.2), not the execution layer. The combination is what makes the system both safe (each run starts clean) and capable (the agent gets a full POSIX environment).

The bi-directional harness — same view, both sides.

A bi-directional harness is the unsexy half of the architecture and the half that determines whether collaboration with an agent feels real or absurd. Dust exposes six shared primitives — Sandbox, Projects, Tasks, Conversations, Files, Tools — and the rule is that both human and agent operate on them.

The principle is simple but rare in practice: whatever the human sees, the agent sees — and vice versa. No private agent state, no human-only side channels. This is what makes pause-for-approval (§1.2) genuinely useful: the human reviewing the paused state is looking at the same object the agent will resume on.

The anatomy of a modern harness — three pillars together.

What's striking is how boring this stack is. Durable workflows have existed since the early 2000s. Sandboxes are well-trodden. The "harness" idea is just symmetric APIs. The novelty isn't in any one piece — it's in the insistence that all three are necessary, and that anyone shipping an agent without all three is shipping a demo.

Pre-release — automated checks plus a human sanity loop.

Pre-release evaluation lives inside the Development phase. The structure splits cleanly into automated and human work, and the automated half splits further by what kind of question you're asking.

The structural point is that human evaluation isn't a final gate — it's a parallel track that calibrates the automated layer. Verifiable evals catch the cheap mistakes; open-ended evals approximate quality; human review keeps the rubric honest. Drop any one of the three and the other two drift.

Post-release — guardrails first, A/B second, monitoring throughout.

The slide showed a single dashed-line container labelled simply Post-release evaluation, with three components flowing into each other.

The order matters. Guardrails first because they protect users from the worst outputs. A/B testing second because it's how new versions actually graduate. Monitoring throughout because it's the only thing that catches slow degradation — the kind that pre-release evaluation, by definition, can't see.

The whole pipeline in one breath

Verifiable + open-ended evals catch the obvious before launch. Human review keeps the rubric calibrated. Guardrails catch the unsafe in production. A/B tests measure whether new versions are actually improvements. Monitoring catches drift. None of these substitutes for any of the others.

Orchestration agents — one front door, a mesh behind.

The slide showed a clean, almost archetypal architecture: a human on the left, real-time user interactions flowing into a single orchestration agent, which fanned out to a dense mesh of domain agents — HR, Sales, IT, Legal, Marketing, Finance, Operations, Customer Support — each containing two or three specialised sub-agents.

How they're connected

The mesh isn't a free-for-all. Three connection types appeared in the legend, and they matter:

• Direct integration — solid lines. Tight coupling, used sparingly.
• Indirect / API calls — dashed lines. The default. Stable contract, loose coupling.
• Data / context flow — dotted lines. Implicit context-sharing, not a control channel.

The orchestration agent's job isn't to do the work. It's to (a) understand what the user is asking, (b) route to the right domain, and (c) hand back a coherent answer when domains have to be combined. That's it. Resist the temptation to make the orchestrator clever.

Batch processing is not good enough anymore — context gets lost.

The slide title was the whole point: "Batch Processing is not good enough anymore in the age of AI as Context gets lost!" The picture underneath showed the typical enterprise data flow — but with the batch lag highlighted as the problem.

Why batch fails for AI specifically

A traditional dashboard is fine with overnight refreshes — the human reading it has their own context. An agent doesn't. When an agent decides at 11:00 based on data refreshed at 02:00, nine hours of operational reality are invisible to it. That gap is where bad agentic decisions live. The fix isn't smarter prompting; it's closing the latency between operational truth and the agent's worldview.

The AI gateway — one abstraction in front of every model.

The pattern is simple enough to describe in one sentence: backend services and local experimentation both call a single AI Gateway, which fans out to whatever model provider is right for the request — Gemini, Anthropic, OpenAI, Mistral, Llama, anyone.

Why this matters more every quarter

The cost of not having a gateway used to be acceptable — you'd hardcode one provider and switch every two years. That stopped being true in 2025. New providers ship every month, models change weekly, and pricing shifts by orders of magnitude. The gateway is the layer that absorbs that volatility so your application doesn't have to.

It's also the layer where you enforce policy: which teams can use which models, who pays for what, what gets logged for compliance. Without a gateway, you discover later that the answers to these questions are scattered across forty repos.

Three predictions worth defending.

Notice what's not in this list: nothing about model capability, AGI timelines, model releases, parameter counts. The bet is that the next twelve months aren't about models getting smarter — they're about the industrial layer around models maturing. Compute mix, pricing model, evaluation discipline.

The one sentence to memorise

The next 12 months — agents become measurable, not magical.

The trick of this sentence is that it forecloses two debates at once. "Magical" is the breathless capability discourse — the next model is going to change everything. "Measurable" is the engineering discourse — the next year is about telemetry, evals, governance, control. The speaker is saying: stop having the first conversation; start having the second.

This is also the centre of gravity for every other point on the day. Polu's harness (§1) is what makes agents governable enough to be measured. The evaluation pipeline (§2) is the measurement. The orchestration mesh (§3) makes agents visible enough to be measured. Predictions one through three (§4.1) describe the economic consequences of measurability — pricing, infrastructure, evals as infra.

Buying agents means buying outcomes — three shifts in the question.

Each of the three shifts pulls power from the vendor's marketing team to the buyer's engineering team. Capability benchmarks live on vendor websites; "does the workflow finish?" lives in the buyer's bug tracker. Demos are theatre; evals on your own tickets are evidence. Accuracy is one number; control is a posture. The whole talk pushes the buyer toward becoming a competent evaluator rather than a passive customer.

Four phrases — print them above your desk.

The four claims interlock. If inference is the workload (not training), then the unit cost that matters is per-inference and pricing follows (talk 4.3). If workloads are long-running, you need durable execution (talk 1.2). If orchestration is the product, then the orchestration layer is where the value accrues (talk 3.1). And if evals shift to trajectories — to whole sequences of model calls in production rather than single-prompt benchmarks — then evaluation becomes infrastructure (talk 4.1).

Four sentences. Twenty words. Most of what was said all day, compressed.