Diagnosing CI And Merge Bottlenecks

1---
2name: diagnosing-ci-and-merge-bottlenecks
3description: >
4  Diagnoses CI and pull-request pipeline health for a GitHub repo using the engineering analytics MCP tools —
5  pull-requests (PR list with CI status), workflow-health (per-workflow CI trends), and pr-lifecycle (a single PR's
6  timeline). Use when asked whether CI is getting faster or slower, which GitHub Actions workflow is the slow or
7  flaky long-pole, how long PRs take from open to merge, how an author's merge time compares to the cohort, which
8  open PRs have failing or pending CI, or where a specific pull request is stuck. Triggers on "engineering
9  analytics", "is CI getting slower", "slow workflow", "flaky CI", "time to merge", "cycle time", "PR throughput",
10  "failing checks", "where is PR <n> stuck", "CI long pole", "what's holding up this PR".
11---
12 
13# Diagnosing CI and merge bottlenecks
14 
15Engineering analytics treats a pull request like product analytics treats a user: a PR moves through a pipeline
16(`opened → CI → review → merged → deployed`) and the job is to find where it slows down. The surface is **three
17named MCP tools** — you call them, you don't write SQL. Dogfooded on `PostHog/posthog`; the same tools serve
18autonomous agents (e.g. PostHog Code) reasoning about their own PRs.
19 
20## The tools
21 
22- **`pull-requests`** — the PR workhorse. Open PRs plus anything merged or closed since `date_from` (default
23  `-30d`), newest first. Each row carries `author` (nested object: `handle`, `display_name`, `is_bot`), `repo`
24  (nested: `owner`, `name`), `state`, `is_draft`, `labels`, `open_to_merge_seconds`, and a `ci` rollup
25  (`runs` / `passing` / `failing` / `pending`) from the head-SHA join. Answers most PR-level questions:
26  which PRs have failing or pending CI, which are stuck open longest, per-author or per-repo triage, and
27  time-to-merge stats (aggregate `open_to_merge_seconds` over the returned merged rows yourself — median and p95,
28  never a mean).
29- **`workflow-health`** — per-workflow CI health over a window (`date_from` / `date_to`, default last 30 days):
30  `run_count`, `success_rate`, `p50_seconds`, `p95_seconds`, `last_failure_at`. Answers "is CI getting faster or
31  slower" and "which workflow is the slow or flaky long pole". There is no built-in trend — call it over two
32  adjacent windows and compare. `success_rate` / `p50_seconds` / `p95_seconds` cover completed runs only and are
33  `null` when a window has no completed runs — guard for null before comparing two windows (a workflow can have
34  runs in one and none in the other).
35- **`pr-lifecycle`** — a single PR's timeline: a header plus ordered events — opened, then a CI started/finished
36  pair **per workflow run** (many on a multi-workflow repo, interleaved by time), then merged/closed. Answers
37  "where is PR N stuck". `metric_quality` is `partial`.
38 
39There is no aggregate time-to-merge tool and no "counts" tool — derive those from `pull-requests` (the stuck/failing
40counts, the merge-time percentiles).
41 
42## Caveats you must carry into every answer
43 
44These are structural limits of today's snapshot data — state them, don't paper over them.
45 
46- **`open_to_merge_seconds` is coarse.** It fuses _draft_ time and _ready-for-review_ time into one figure. Report
47  it as "open to merge", never "cycle time" or "review time". Flag it when long-lived drafts inflate a number.
48- **CI status can be stale.** The CI source syncs on a watermark and does not refresh a run that completes after
49  newer runs land (until the `workflow_run` webhook ships). Treat a `pending` count as unsettled, not as a settled
50  failure; lead with status, not a verdict.
51- **CI for a PR is the head-SHA join, nothing else.** The `ci` rollup reflects only the latest commit's runs. There
52  is no other link between a PR and its checks.
53- **No reviews, approvals, per-check/job, or deploys yet.** Don't infer review behaviour or DORA metrics from their
54  absence; that data hasn't landed. `pr-lifecycle` is `partial` for the same reason.
55- **Bots and drafts are present in `pull-requests` output, excluded by convention.** Filter out `author.is_bot`
56  (nested under `author`, not a row-level field) and `is_draft` for throughput / merge-time questions; keep them in
57  for bot-impact questions.
58- **`pull-requests` returns a capped page.** At most `limit` rows (newest first); `truncated` is `true` when more
59  match, and there is no repo or limit filter to narrow the call. When `truncated` is `true`, any percentile or
60  count you derive covers only the newest page — not the whole window — so say so and shrink `date_from` until the
61  real set fits under the cap.
62 
63## Choosing a tool
64 
65| The question                                           | Tool              | How                                                                                                                                                                                                                                                                                            |
66| ------------------------------------------------------ | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
67| Is CI getting slower? Which workflow is the long pole? | `workflow-health` | Call over two adjacent windows (e.g. `date_from=-14d`, then `date_from=-28d` `date_to=-14d`); compare `p50_seconds` and `p95_seconds` per workflow. Lead with the median but always check p95 separately — they move independently.                                                            |
68| Which open PRs have failing or pending CI?             | `pull-requests`   | Keep rows where `ci.failing > 0` or `ci.pending > 0`. `pending` means unsettled (or stale) — not a settled failure.                                                                                                                                                                            |
69| Which PRs are stuck open longest?                      | `pull-requests`   | Keep `state = open`, not `is_draft`, not `author.is_bot`; sort by `created_at` ascending (oldest first).                                                                                                                                                                                       |
70| How long are PRs taking to merge? Per author?          | `pull-requests`   | Over merged rows (`merged_at` set, not bot, not draft), aggregate `open_to_merge_seconds` — median and p95. Group by `author.handle` for **cohort context, not a ranking** (per-developer surveillance is an explicit non-goal). Trend it by calling with two `date_from` windows.             |
71| Where is PR N stuck?                                   | `pr-lifecycle`    | Walk the sorted events: `opened → first CI started`, the CI span (first start → last finish; one pair per workflow), `last CI finished → merged`. The largest gap is the bottleneck. A long open→merge with quick CI points at review/idle time the `partial` data can't itemize yet — say so. |
72 
73## The high-value chain
74 
75Mirror how a human investigates: aggregate signal → confirm → concrete PR.
76 
77```text
78workflow-health  (find the slow/flaky long-pole workflow)
79   → pull-requests  (confirm it's dragging merge time; list the affected PRs)
80      → pr-lifecycle  (open a representative stuck PR and show the gap)
81```
82 
83"CI median rose because `e2e-playwright` p95 doubled; that workflow is the long pole on PR #1234, which sat 47m in
84CI before merging."
85 
86## Output expectations
87 
88- Lead with the verdict in one line, then the supporting numbers.
89- Carry the coarse / partial / staleness caveat whenever the distinction matters.
90- For multi-window or multi-workflow comparisons, a short table beats prose. Report median and p95 side by side —
91  never collapse them into one "average".
92 
93## What NOT to do
94 
95- Don't call `open_to_merge_seconds` cycle time or review time — it's coarse open-to-merge.
96- Don't report a CI count as a settled failure when `pending > 0` — it may be unsettled or stale.
97- Don't infer reviews, approvals, per-check counts, or deploys — that data isn't ingested yet.
98- Don't turn per-author buckets into a leaderboard — they're for finding stuck work, not ranking people.
99- Don't reach for these tools to fetch raw PR contents or diffs — they surface pipeline signal, not the PR thread.
100