A/B Test Memory

​

How it works

1. Reads your current metadata from user_apps (web Supabase mirror of ASC).
2. Computes a stable fingerprint over title + subtitle + keywords + description + promotional text + what’s new.
3. Compares against the previous snapshot in cofounder_metadata_snapshots.
4. If the fingerprint changed → records the diff and schedules a 14-day follow-up.
5. When the follow-up window has elapsed → measures impressions / conversion / downloads in the 14 days before vs after.

​

What you see (fresh change)

## 🧪 Metadata Change Detected (3d ago, v3.2.1)
  • subtitle: "Take amazing screenshots" → "AI-powered screenshot stitching"
  • keywords: "screenshots,photos,export" → "screenshots,stitching,ai,photos"

(Scheduled 14-day follow-up to measure impact.)

​

What you see (after 14 days)

## 🧪 Metadata Change Detected (14d ago, v3.2.1)
  • subtitle: "Take amazing screenshots" → "AI-powered screenshot stitching"
  • keywords: "screenshots,photos,export" → "screenshots,stitching,ai,photos"

Measured impact (14d window):
  Impressions/day: 547 → 891
  Conversion:      0.6% → 1.2%
  Downloads/day:   3 → 11

​

In the structured report

{
  "changeDate": "2026-04-19",
  "field": "subtitle, keywords",
  "observedImpact": "Impressions +63%, conversion 0.6→1.2%, downloads 3→11/day.",
  "conclusion": "worked"
}

​

Why this matters

• “Last time we shipped a subtitle change, conversion went 0.6 → 1.2%. This new draft might do similar.”
• “Two months ago we tried ‘AI’ in the subtitle and it regressed downloads. Don’t propose it again.”

​

Edge cases

• Multiple changes in a window — only the most recent change is treated as the “experiment”. Earlier ones are still recorded but the impact attribution fuzzes.
• External factors — the agent acknowledges seasonality / category shifts in its prose when the measured impact is suspicious.
• Missing analytics — if ASC analytics are absent, the impact field is null and the agent says so plainly instead of inventing numbers.