Skip to main content

​
Evals

Every applied action records a prediction (its expected impact) and, after the outcome window, a measurement. Evals surfaces that closed loop so you can see whether the agent’s calls actually pan out — the visible side of its self-learning.

​
What it shows

  • Win rate — of the decided experiments, how many worked (won) vs. underperformed (lost). Neutral outcomes count as decided but not a win.
  • By category — win rate per kind (ASO metadata, product, pricing, release, retention, PPO…).
  • Predicted → measured ledger — each experiment with its hypothesis (predicted) and the measured outcome (✅ worked / ❌ underperformed / ➖ no change / ⏳ measuring), including the per-day delta where available.

​
Where the data comes from

  • Predictions — cofounder_experiments (hypothesis + expected metrics), written when an action is applied.
  • Measurements — cofounder_experiment_observations, written by the daily measurement crons: causal for visual/PPO (Apple’s A/B result), correlational before/after for metadata/product/pricing/release/retention.
This is the same signal that feeds the agent’s calibration — the agent leans into categories that tend to work and pulls back from ones that don’t.

​
UI & API

Co-founder → Evals. Or:
Returns totals (won/lost/neutral/tooEarly), winRatePct, a byCategory breakdown, and the detailed rows ledger.

​
See also