Action Scoring & Self-Learning
The agent doesn’t just propose actions — it ranks them by a computed score, measures whether they worked, and learns which kinds of action land. This is the machinery behind the Evals view.Structured impact
Every prioritized action carries a humanestimatedImpact string and a
machine-readable twin the model fills:
Deterministic score
Ranking is computed in code, not by the model’s ordering — so it’s stable and not at the mercy of small-model drift:priorityScore is written onto each action after synthesis (the model leaves
it null). Actions with an unmet dependency (see below) are demoted so
prerequisites out-rank the work they unblock.
Sequencing & grouping
Related actions share agroup label; a later action lists the groups it
dependsOn. The scorer keeps prerequisites ahead, and clients render the
sequence.
Closed-loop verification
When an action is applied, its prediction is recorded and — after the horizon — measured:- Causal for visual/PPO changes (Apple’s A/B traffic-split result).
- Correlational before/after ASC metrics for metadata / product / pricing / release / retention.
won / lost / neutral) surface in Evals
and close the linked memory episode so reflection distills a
fact.
Calibration (two-way)
The old loop only learned from rejection. Now positive signal counts too: when you approve/apply an action, a per-category accept-rate is tracked, and combined with the win-rate from closed-loop measurement into a calibration multiplier on the score — plus a “preference profile” injected into the agent’s context. Net effect: categories you act on and that tend to work get boosted; ones you reject or that flop get damped.Staleness
Aproposed action the agent stops re-proposing is retired to superseded
after ~14 days, so the Actions queue reflects what’s still live.
See also
- Evals — the predicted-vs-measured view
- Autopilot Scorecard — trust surface
- Decision Log — the actions ledger