Published: 2026-09-26
Deep dive

Long-horizon agents need experiments, not prompts: a five-step auto-research loop with scorecards

Chapters / key moments (click to jump — plays here on the page)

Erina Karati (ex-Microsoft, Supercell AI Innovation Lab) built Project Paradox, a framework of stateful game agents. It worked in short sessions and drifted over long ones. Her fix is a Karpathy-style auto-research loop around the agents. It carries over to any agent that keeps state, including memory-heavy personal agents like Hermes.

Source video

"Long-Horizon Agents Need Experiments, Not Just Prompts — Erina Karati" by AI Engineer — Watch on YouTube →

The architecture that worked short-term

Per-agent memory namespaces (RAG-backed), so memories don't bleed between agents.
An emotion vector (joy, sadness, fear, anger, disgust) updated after events.
Belief/trust scores toward other agents and the player, raised or lowered by the LLM after each interaction.
Importance scores on memories: anything above a threshold goes to a separate cache for better retrieval later.

Where it broke

Over long horizons, social consistency decayed. Agents kept the topic but lost the source, rumours hardened into stated facts, and agents knew a fact but didn't use it when planning. More memory didn't fix this. Agents need provenance (firsthand vs secondhand, verified vs uncertain), and raw episodic memory should be kept separate from current beliefs.

The recipe

Freeze the harness, the scenarios and the metrics. The optimiser may not touch them.
Define controlled scenarios, for example: public fact diffusion ("the bakery closes tomorrow" — who learns it, do they remember who said it?), rumour uncertainty (does "might leave" become "is leaving"?), replanning (a route gets blocked — do agents tell each other?).
Log structured traces: observations, conversations, memory writes, retrievals, belief updates.
Score with a balanced scorecard, never one "agent quality" number. Measure diffusion reach, source retention, uncertainty preservation and false-certainty rate, action consistency and time to replan, and privacy containment. Optimising one metric alone games it: diffusion alone teaches oversharing, and recall alone surfaces stale memories.
Expose a small policy surface and search over it: memory-write policy, retrieval policy, communication prompt, trust rules, source attribution, replanning triggers. Keep a change only if the scorecard improves and the guardrails hold. Otherwise revert (a ratchet).

Examples of the kind of change the loop proposes: preserve source in memory writes and summaries; store confidence, mark firsthand vs secondhand, require hedging when retelling; classify useful public facts so agents proactively share them.

Why it matters beyond games

Support agents need to know which policy update supersedes which. Personal assistants need to keep, and correct, commitments. Research agents need provenance and contradiction handling. Coding agents need context across issues and changing requirements. They all carry state that affects future actions. She is careful about the claim: this is the right kind of surface to optimise, not proof that the system improved in general. Related reading: how Hermes memory works, context window.

What you can actually set up from this

Extracted from the video's own transcript — the specifics the original summary left out.

Reproducible steps

Freeze the harness, scenarios and metrics
The optimiser should not be able to rewrite the whole codebase or the evaluation. Only a small, named policy surface stays editable: memory writing policy, retrieval policy, communication prompt, belief and trust rules, source attribution, and replanning triggers.
Define controlled scenarios
Examples: a public fact spreads (does the right set of agents learn it and remember who said it), a rumour spreads (does 'might leave' stay uncertain or become 'is leaving'), and a blocked route (do agents replan and tell each other).
Log full traces for each run
Observations, conversations, memory writes, retrievals and belief updates are collected so the run can be compared with the scenario's ground truth.
Score with a balanced scorecard
Reach (how many agents know the fact after N steps), source retention, uncertainty preservation and false-certainty rate for rumours, action consistency and time to replan for planning, and containment for privacy.
Let an outside optimiser propose one small change, then ratchet
The auto-research layer sits outside the system, proposes a constrained policy change, reruns the scenario and keeps the change only if the scorecard improves and the guardrails hold; otherwise it reverts.

Gotchas

A single 'agent quality' number hides failures. Optimising only for diffusion makes agents overshare; optimising only for recall produces noisy or stale memories.
Rollback is not optional: a policy that spreads public facts faster may also leak private information, and one that raises recall can increase stale-memory use.
Memory alone is not enough. Agents need to know where information came from, whether it was first-hand, second-hand, verified or uncertain, and raw episodic memory should be kept separate from current beliefs.
The speaker is careful about claims: without repeated loop results he does not say the system generally improved, only that this is the right kind of surface to expose to such a loop. The one reported example is a rumour about mangoes that was answered correctly after a loop.
The agent design in the talk: per-agent RAG memory namespaces, an emotion vector (joy, sadness, fear, anger, disgust), trust scores toward other agents and the player, and an importance score that decides which memories go into a separate cache.