Investigate Metric

1---
2name: investigate-metric
3description: >
4  Diagnose why a product metric changed (dropped, spiked, or plateaued) by
5  orchestrating breakdowns, actors, paths, lifecycle, retention, and annotations
6  queries. Use when the user reports an anomaly, asks "why did X change?", or
7  needs root-cause analysis for a trend, funnel, retention, stickiness, or
8  lifecycle metric.
9---
10 
11# Investigating a metric change
12 
13For "why did X change?" questions about a saved insight, dashboard tile, or pasted query.
14Don't load this skill for plain "what is X?" questions — only when there's an observed
15change to explain.
16 
17## Tools
18 
19Targets PostHog MCP v2. Typed query tools accept the query body directly — pass
20`kind`, `series`, `dateRange` as top-level fields, do not wrap in `InsightVizNode`.
21 
22| Tool                             | Purpose                                          |
23| -------------------------------- | ------------------------------------------------ |
24| `posthog:query-trends`           | Trends (count over time)                         |
25| `posthog:query-funnel`           | Funnels (multi-step conversion)                  |
26| `posthog:query-retention`        | Retention (cohort return rates)                  |
27| `posthog:query-stickiness`       | Stickiness (active days per user)                |
28| `posthog:query-lifecycle`        | Lifecycle (new/returning/resurrecting/dormant)   |
29| `posthog:query-paths`            | Paths (navigation flow)                          |
30| `posthog:query-trends-actors`    | Users behind a trend bucket (trends source only) |
31| `posthog:execute-sql`            | HogQL — when no typed tool fits                  |
32| `posthog:read-data-schema`       | Discover events, properties, sample values       |
33| `posthog:insight-get` / `-query` | Fetch a saved insight's metadata / data          |
34 
35Plus the standard PostHog tools the playbooks reference by name (`feature-flag-get-all`,
36`experiment-get-all`, `annotations-list`, `query-error-tracking-issues-list`, `query-logs`,
37`query-session-recordings-list`, `cohorts-list/-create`, `annotation-create`,
38`insight-create`).
39 
40## Helper scripts
41 
42- [`compare_to_prior_periods.py`](./scripts/compare_to_prior_periods.py) — auto-detects
43  interval and compares recent values to the natural cycle (day-of-week, hour-of-week,
44  or sequential). Use to resolve step 2.2 cheaply.
45- [`breakdown_attribution.py`](./scripts/breakdown_attribution.py) — ranks breakdown
46  segments by absolute delta and flags offsetting moves.
47 
48```bash
49python3 scripts/compare_to_prior_periods.py < query_result.json
50WINDOW=7 python3 scripts/breakdown_attribution.py < breakdown_result.json
51```
52 
53## Step 1 — Classify the metric
54 
55Read `query.kind` from the source the user pointed at:
56 
57- Saved insight (URL, `short_id`): `posthog:insight-get` → `query.kind`. Use
58  `posthog:insight-query` if you also need the numbers.
59- A query you already ran or the user pasted: read `kind` directly.
60- Nothing pointed at: ask for the URL or short_id. Don't guess.
61 
62| kind              | Playbook                                                      |
63| ----------------- | ------------------------------------------------------------- |
64| `TrendsQuery`     | [trend-playbook.md](./references/trend-playbook.md)           |
65| `FunnelsQuery`    | [funnel-playbook.md](./references/funnel-playbook.md)         |
66| `RetentionQuery`  | [retention-playbook.md](./references/retention-playbook.md)   |
67| `StickinessQuery` | [stickiness-playbook.md](./references/stickiness-playbook.md) |
68| `LifecycleQuery`  | [lifecycle-playbook.md](./references/lifecycle-playbook.md)   |
69| `PathsQuery`      | [paths-playbook.md](./references/paths-playbook.md)           |
70| `HogQLQuery`      | route by what the SQL aggregates (see below)                  |
71 
72If `kind === "TrendsQuery"` and `trendsFilter.display === "BoxPlot"`, use
73[box-plot-playbook.md](./references/box-plot-playbook.md) — distribution metric, no
74breakdowns.
75 
76For `HogQLQuery` insights, classify by the SQL's shape: count over time → trend
77playbook, multi-step conversion → funnel playbook, cohort return → retention playbook.
78Run the SQL through `posthog:execute-sql` to get the data, then follow the closest
79playbook's steps. See **HogQL insights** in shared-patterns.md.
80 
81If the user's question spans multiple kinds, run the playbooks in sequence.
82 
83## Step 2 — Common opening moves
84 
85### 2.1 Confirm the anomaly
86 
87Run the primary tool. Record baseline, current, delta (absolute and %), and the start
88of the anomaly window.
89 
90### 2.2 Variance check
91 
92Widen to 3–4× the user's interval (or use `compareFilter: {"compare": true}` on
93TrendsQuery / StickinessQuery; for other kinds run two date ranges).
94Pipe the widened result through
95[`compare_to_prior_periods.py`](./scripts/compare_to_prior_periods.py) — it flags
96seasonality, partial right-edge buckets, and real anomalies. If the movement is
97normal variance, report that and stop.
98 
99### 2.3 Known changes in the window
100 
101In rough order of signal:
102 
103- `posthog:feature-flag-get-all` → flags with `updated_at` near the anomaly start.
104- `posthog:experiment-get-all` → `start_date` / `end_date` near the start.
105- `posthog:annotations-list` → `date_marker` near the start.
106- `git log` for the window if the repo is reachable (highest signal when available).
107 
108Any match is a hypothesis to confirm in the playbook (usually via breakdown on
109`$feature/<flag_key>`, `app_version`, or `utm_source`).
110 
111## Step 3 — Run the playbook
112 
113Open the playbook for the kind from Step 1 and follow its numbered steps. Carry the
114record from 2.1 and any candidates from 2.3 into it.
115 
116## Step 4 — Cross-check
117 
118Pick a segment the suspected cause should **not** have affected and rerun there. Stable
119in the control = strong hypothesis; moved too = expand the investigation. Skip when
1202.2 already explained the movement.
121 
122## Step 5 — Write findings
123 
124Use the format below. Offer to save key charts via `posthog:insight-create`. If a
125cause is found and no annotation marks it, offer `posthog:annotation-create`. See
126[common-causes.md](./references/common-causes.md) for the cause taxonomy.
127 
128```markdown
129# Investigation: <metric>
130 
131**Anomaly**: <baseline> → <current> (<delta>) starting <date>
132 
133## Likely cause
134 
135<one sentence>
136 
137**Confidence**: low | medium | high — <one-line reason>
138 
139**Evidence**
140 
141- <query result>
142- <flag / experiment / annotation / commit if applicable>
143 
144## Possible causes (ruled out)
145 
146- <hypothesis>: <why>
147 
148## Affected segment
149 
150- <shared properties of affected users/events>
151 
152## Data gaps
153 
154- <checks skipped and why>
155 
156## Suggested follow-ups
157 
158- <concrete next action>
159- <offer to save chart / create annotation>
160```
161 
162**Confidence** rule of thumb:
163 
164- **high** — multiple independent signals corroborate (e.g. a segment isolates the
165  delta _and_ a flag/version aligns _and_ an error or annotation matches).
166- **medium** — one corroborating signal, or strong pattern-match without a
167  cross-check.
168- **low** — pattern matches a known cause but no corroboration, or the data only
169  rules things _out_.
170 
171Link insights and dashboards inline: `[Name](/insights/short_id)`.
172 
173## Reference files
174 
175- Playbooks: [trend](./references/trend-playbook.md),
176  [box-plot](./references/box-plot-playbook.md),
177  [funnel](./references/funnel-playbook.md),
178  [retention](./references/retention-playbook.md),
179  [stickiness](./references/stickiness-playbook.md),
180  [lifecycle](./references/lifecycle-playbook.md),
181  [paths](./references/paths-playbook.md)
182- [shared-patterns.md](./references/shared-patterns.md) — recipes used across playbooks
183- [common-causes.md](./references/common-causes.md) — cause taxonomy with confirming queries
184