Auditing Experiments Flags

1---
2name: auditing-experiments-flags
3description: 'Audit PostHog experiments and feature flags for configuration issues, staleness, and best-practice violations. Read when the user asks to audit, health-check, or review experiments or feature flags, check flag hygiene, or verify experiment setup.'
4---
5 
6# Auditing experiments and feature flags
7 
8This skill teaches you how to run configuration audits on experiments and feature flags.
9All checks use the experiment and feature flag read tools (`experiment-get`, `experiment-list`, `feature-flag-get-definition`, `feature-flag-get-all`) — no SQL queries are needed for Phase 1 checks.
10 
11## Usage modes
12 
13### Quick check (single entity)
14 
15When the user asks about a specific experiment or flag:
16 
171. Fetch the entity via `experiment-get` (experiment ID) or `feature-flag-get-definition` (numeric flag ID).
182. Apply the relevant checks from [experiment checks](./references/experiment-checks.md) or [flag checks](./references/flag-checks.md).
193. Report findings inline as markdown, grouped by severity (CRITICAL first, then WARNING, then INFO).
204. Include entity links as `[Experiment: name](/experiments/id)` or `[Flag: key](/feature_flags/id)`.
21 
22### Scoped audit (one domain)
23 
24When the user asks to audit all experiments or all flags:
25 
261. Bulk-fetch via `experiment-list` or `feature-flag-get-all`.
272. Run all checks for that domain against each entity.
283. Group findings by severity, then by entity.
294. Report as inline markdown.
30 
31### Full audit (comprehensive)
32 
33When the user asks for a comprehensive audit of both experiments and flags:
34 
351. Fetch all experiments via `experiment-list` and all flags via `feature-flag-get-all`.
362. Run all experiment checks and all flag checks.
373. Apply [recurring patterns](./references/synthesis-patterns.md) to identify patterns across multiple findings.
384. If there are more than 5 entities with findings, output as a notebook artifact via `notebooks-create` for easier navigation. Otherwise report inline.
39 
40## Output format
41 
42For each finding, include:
43 
44- **Severity badge**: `🔴 CRITICAL`, `🟡 WARNING`, or `🔵 INFO`
45- **Check name**: Which check produced this finding
46- **Entity link**: Markdown link to the entity
47- **What's wrong**: One-sentence description
48- **Action**: What to do about it (see [remediation actions](./references/remediation-actions.md))
49 
50Example:
51 
52> 🟡 **WARNING** — Flag integration · [Experiment: checkout-redesign](/experiments/42)
53> The linked feature flag is inactive (paused). Traffic is not being split.
54> **Action**: Re-enable the flag or end the experiment.
55 
56## Handling unavailable data
57 
58Some checks require activity logs (`feature-flags-activity-retrieve` for flags), which may not be available in every session.
59If activity log data is unavailable:
60 
61- Skip `checkActivityHistory` (experiment check) entirely.
62- Skip the "toggle instability" and "never activated" sub-checks in flag lifecycle checks.
63- In your report, note which checks were skipped and why:
64  > _Skipped: Activity history checks (activity logs not available via current tools)_
65 
66## Partial failures
67 
68If a fetch call fails for some entities:
69 
70- Continue with the entities you could fetch.
71- Report which entities could not be assessed and why.
72- Do not silently omit entities from the audit.
73 
74## Reference files
75 
76- [Experiment checks](./references/experiment-checks.md) — experiment configuration checks
77- [Flag checks](./references/flag-checks.md) — feature flag checks
78- [Finding types](./references/finding-taxonomy.md) — severity and category definitions
79- [Recurring patterns](./references/synthesis-patterns.md) — patterns across multiple findings
80- [Remediation actions](./references/remediation-actions.md) — what to do about each finding
81