Managing Experiment Lifecycle

1---
2name: managing-experiment-lifecycle
3description: "Guides experiment state transitions: launching, pausing, resuming, ending, shipping variants, archiving, resetting, duplicating, and copying to another project. Covers preconditions, implications for variant assignment and analysis, and the decision framework for when to use each action.\nTRIGGER when: user asks to launch, pause, resume, end, ship, archive, reset, duplicate, or copy an experiment to another project.\nDO NOT TRIGGER when: user is creating an experiment (use creating-experiments), configuring rollout (use configuring-experiment-rollout), or setting up metrics (use configuring-experiment-analytics)."
4---
5 
6# Managing experiment lifecycle
7 
8This skill covers experiment state transitions — what each action does, when to use it, and how it affects variant assignment and analysis.
9 
10## State diagram
11 
12```text
13draft ──launch──▶ running ──end──▶ stopped ──archive──▶ archived
14                    │   ▲              │
15                  pause resume    ship_variant
16                    │   │         (also ends if running)
17                    ▼   │
18                  paused (flag inactive, still "running" status)
19 
20Any non-draft state ──reset──▶ draft
21```
22 
23## Actions and their implications
24 
25For each action, the two key questions:
26 
271. **Who sees what variant?** (user perspective)
282. **Who is in my analysis?** (statistical perspective)
29 
30### Launch (`experiment-launch`)
31 
32Transitions draft → running. Activates the feature flag and sets `start_date`.
33 
34- **Preconditions**: must be in draft, flag needs ≥2 variants with "control" first
35- **Pre-launch checklist**: has at least one metric? Variants correct? Flag implemented in code?
36- **Variants**: users start being bucketed into variants based on the configured split
37- **Analysis**: data collection begins from `start_date`
38 
39No request body needed.
40 
41### Pause (`experiment-pause`)
42 
43Deactivates the feature flag. Users fall back to the default experience (typically control).
44 
45- **Preconditions**: must be running and not already paused
46- **Variants**: flag is not returned by `/decide` — no new exposure events recorded
47- **Analysis**: no new data while paused, but existing data is preserved. Experiment stays "running".
48 
49No request body. Use `experiment-resume` to reactivate.
50 
51### Resume (`experiment-resume`)
52 
53Reactivates the feature flag after a pause. Users are re-bucketed deterministically into the same variants.
54 
55- **Preconditions**: must be paused
56- **Variants**: same assignment as before pause — deterministic bucketing
57- **Analysis**: exposure tracking resumes
58 
59No request body.
60 
61### End (`experiment-end`)
62 
63Sets `end_date` and transitions to stopped. The feature flag is **NOT modified**.
64 
65- **Preconditions**: must be running (launched, not already stopped)
66- **Variants**: users continue seeing assigned variants (flag stays active)
67- **Analysis**: results frozen to data up to `end_date`
68 
69Optional body: `conclusion` ("won", "lost", "inconclusive", "stopped_early", "invalid") and `conclusion_comment`.
70 
71Use this when you want to freeze results without changing what users see.
72 
73### Ship variant (`experiment-ship-variant`)
74 
75Rewrites the feature flag so the selected variant is served to 100% of users.
76 
77- **Preconditions**: must be launched (running or stopped). Cannot ship from draft.
78- **Variants**: ALL users see the shipped variant. The flag is rewritten with a catch-all group.
79- **Analysis**: if still running, the experiment is also ended (end_date set)
80 
81**Always confirm with the user before shipping** — this permanently rewrites the feature flag.
82 
83Required: `variant_key` (e.g. "test"). Optional: `conclusion`, `conclusion_comment`.
84 
85Returns 409 if an approval policy requires review before the flag change.
86 
87### Archive (`experiment-archive`)
88 
89Hides a stopped experiment from the default list view.
90 
91- **Preconditions**: must be stopped (end_date set)
92- **Variants**: no change — flag is unaffected
93- **Analysis**: no change — results remain accessible
94 
95No request body. Can be restored by setting `archived=false` via `experiment-update`.
96 
97### Reset (`experiment-reset`)
98 
99Returns an experiment to draft state. Clears `start_date`, `end_date`, `conclusion`, and `archived`.
100 
101- **Preconditions**: must not already be in draft
102- **Variants**: flag is left unchanged — users continue seeing assigned variants
103- **Analysis**: previously collected data still exists but won't be included in results unless `start_date` is adjusted after re-launch
104 
105No request body.
106 
107### Duplicate (`experiment-duplicate`)
108 
109Creates a copy as a new draft with fresh dates and no results.
110 
111**Important**: always provide a unique `feature_flag_key` different from the original. If the same key is used, both experiments share a flag — changes to one affect both.
112 
113Optional: custom `name` (defaults to "Original Name (Copy)").
114 
115### Copy to project (`experiment-copy-to-project`)
116 
117Copies an experiment into a **different project in the same organization** as a new draft. Use this instead of
118`experiment-duplicate` when the copy should land in another project; use duplicate when it stays in the same project.
119 
120- **Preconditions**: source must not use legacy metrics; target project must be in the same organization and you must
121  have write access to it. Cannot copy across organizations or regions.
122- **What's copied**: name, description, type, parameters, filters, primary/secondary metrics (fresh uuids), stats and
123  scheduling config, exposure criteria. **Not copied**: saved-metric references (project-scoped), holdout, exposure
124  cohort, dates, results, conclusion.
125- **Feature flag**: `target_team_id` is required; `feature_flag_key` is optional. The resolved key is then looked up
126  **in the target project**, and the lookup result — not whether you passed the key — decides what happens:
127  - **If `feature_flag_key` is omitted**: it defaults to the _source_ experiment's flag key. That key normally
128    doesn't exist in the target project, so a new flag with it is created there. (The default can still collide — see
129    the next point — so to be safe, pass an explicit key.)
130  - **If the resolved key already exists as a flag in the target project**: the copy **shares** that existing flag
131    instead of creating one. Both experiments then point at the same flag, so lifecycle ops (ship, pause) on either
132    affect both. The existing flag must have ≥2 variants including one keyed `control`, otherwise the call returns 400.
133  - **If the resolved key does not exist in the target project**: a new, independent flag is created with that key.
134    To guarantee independence, pass a `feature_flag_key` that doesn't already exist in the target.
135 
136**Confirm the source experiment and target project by name before calling** — this writes into a project the user
137isn't looking at. The returned experiment (and its id) belongs to the target project.
138 
139## Decision framework
140 
141| Situation                                          | Action                   | Tool                         |
142| -------------------------------------------------- | ------------------------ | ---------------------------- |
143| Draft ready, flag implemented, metrics set         | Launch                   | `experiment-launch`          |
144| Clear winner, significant results                  | Ship the winning variant | `experiment-ship-variant`    |
145| No significant difference after sufficient time    | End as inconclusive      | `experiment-end`             |
146| Something wrong, need to stop exposure temporarily | Pause                    | `experiment-pause`           |
147| Resume after pause                                 | Resume                   | `experiment-resume`          |
148| Experiment ended, ready to clean up                | Archive                  | `experiment-archive`         |
149| Need to start over with same config                | Reset to draft           | `experiment-reset`           |
150| Want a similar experiment with a fresh start       | Duplicate                | `experiment-duplicate`       |
151| Want the same experiment in a different project    | Copy to another project  | `experiment-copy-to-project` |
152 
153## Resolving experiments
154 
155All lifecycle actions require an experiment ID. If you don't have one, load the
156`finding-experiments` skill to resolve the user's reference (name, description,
157"latest", etc.) to a concrete ID before proceeding.
158 
159## Error handling
160 
161| Error message                           | Meaning                              |
162| --------------------------------------- | ------------------------------------ |
163| "Experiment has already been launched." | Can't launch a non-draft experiment  |
164| "Experiment has not been launched yet." | Can't end/pause/ship a draft         |
165| "Experiment has already ended."         | Can't end/pause a stopped experiment |
166| "Experiment is already paused."         | Use resume instead                   |
167| "Experiment is not paused."             | It's already active                  |
168| "Experiment is already in draft state." | Nothing to reset                     |
169| "Experiment is already archived."       | Already done                         |
170 
171When you get a 400, explain the situation to the user rather than retrying.
172