Creating Experiments

1---
2name: creating-experiments
3description: "Guides agents through the 3-step experiment creation flow: defining the hypothesis, configuring rollout, and setting up analytics. Delegates rollout decisions to configuring-experiment-rollout and metric setup to configuring-experiment-analytics.\nTRIGGER when: user asks to create a new experiment or A/B test, OR when you are about to call experiment-create.\nDO NOT TRIGGER when: user is updating an existing experiment, managing lifecycle, or only browsing experiments."
4---
5 
6# Creating experiments
7 
8This skill walks through the 3-step flow for creating a new A/B test experiment.
9 
10## Core principle: draft first, iterate on details
11 
12Create the experiment as a draft quickly, then iterate on metrics and configuration.
13The user gets a tangible draft immediately and can refine it.
14 
15## The 3-step creation flow
16 
17### Step 1: What are we testing?
18 
19Gather these before calling `experiment-create`:
20 
21- **Experiment name** — descriptive, inferred from context when possible
22- **Hypothesis** — what you expect to happen (goes in `description`)
23- **Feature flag key** — kebab-case. Ask if they want a new flag or to reuse an existing one.
24  The flag is auto-created — do NOT create one separately.
25- **Type** — leave empty (will internally default to `"product"`. The `"web"` value is reserved for no-code experiments configured visually with the PostHog
26  toolbar in a browser; it cannot be meaningfully driven via MCP. If a user asks for a
27  no-code/toolbar experiment, point them to the PostHog UI instead of creating one here.)
28 
29If the user gives enough context to infer these, don't ask — just proceed.
30 
31### Step 2: Who sees what variant?
32 
33This is about rollout configuration.
34 
35**Before asking any rollout question, load `configuring-experiment-rollout`.** The disambiguation wording, recommendations, and post-answer branches live there — do not formulate rollout questions yourself, and do not assume an example you remember covers the user's path.
36 
37Key decision points (covered in detail by `configuring-experiment-rollout`):
38 
39- Variant split (how many variants, what percentage each)
40- Overall rollout percentage (what % of all users enter the experiment)
41- Whether to persist the flag across authentication steps
42 
43If the user doesn't mention rollout specifics, use defaults: 50/50 control/test, 100% rollout.
44 
45### Step 3: How to measure impact?
46 
47This is about analytics and metrics. **Load the `configuring-experiment-analytics` skill** for guidance.
48 
49**Do NOT configure metrics on creation.** Metrics are not passed to `experiment-create` — they are added
50afterwards via `experiment-update`. This keeps the creation call lightweight.
51 
52When the user specifies metrics upfront, acknowledge them and add them immediately after creation.
53When they don't, create the draft and then guide them through metric setup as a follow-up.
54 
55## How to create
56 
57Call `experiment-create` with:
58 
59```json
60{
61  "name": "Descriptive experiment name",
62  "feature_flag_key": "kebab-case-key",
63  "description": "Hypothesis: [what you expect to happen]",
64  "parameters": {
65    "feature_flag_variants": [
66      { "key": "control", "name": "Control", "split_percent": 50 },
67      { "key": "test", "name": "Test", "split_percent": 50 }
68    ],
69    "rollout_percentage": 100
70  }
71}
72```
73 
74Two different percentages — do NOT mix them up:
75 
76- `feature_flag_variants[].split_percent` — how users **inside** the experiment are split across variants (must sum to 100, recommended to have an even split).
77- `parameters.rollout_percentage` — what fraction of **all** users enter the experiment at all (0-100, defaults to 100).
78 
79Key details:
80 
81- First variant must have key `"control"`. Minimum 2, maximum 20 variants.
82- `rollout_percentage` defaults to 100 if omitted.
83- Stats default to Bayesian. Only set `stats_config` if the user requests Frequentist.
84 
85## After creation
86 
871. **Always show the experiment URL.** The `experiment-create` response includes `_posthogUrl` — always display this link so the user can view and configure the experiment in the UI.
88 
892. **Remind the user to implement the feature flag in code.** Link to the experiment page and say "implement the flag as shown here" — the experiment detail page shows implementation snippets for the user's SDK.
90 
913. **Guide through metrics** if not yet configured — load the `configuring-experiment-analytics` skill.
92 
934. **Launch** when ready — use the `experiment-launch` tool.
94