Grouping Noisy Errors

1---
2name: grouping-noisy-errors
3description: >
4  Consolidate PostHog error tracking issues that are the same actual
5  error reported under different fingerprints. Use when the user asks
6  "why do I have so many TypeError issues that look the same?", "merge
7  these duplicates", "stop splitting this error into new issues", or
8  wants to clean up fingerprint sprawl. Decides between a one-shot merge
9  of existing issues and a durable grouping rule that keeps future
10  events from creating new fingerprints. Does NOT group conceptually
11  similar bugs across different runtimes, SDKs, or call sites.
12---
13 
14# Grouping noisy errors
15 
16The same error can be reported as dozens of separate issues when stack frames or
17messages contain volatile data — random IDs, dynamic file paths, build hashes,
18anonymous function names. The fix is two-step: merge the existing issues into one
19target, then create a grouping rule so future events from the same call site
20share a single canonical fingerprint instead of spawning new ones.
21 
22Important up front: "same error" here is narrow. Two issues that share a name or
23a sentence of message text but came from different code paths, different SDKs,
24or different runtimes are **different errors** and should stay separate, even if
25the user thinks of them as "the same kind of bug". Grouping a frontend
26`TypeError` together with a backend `TypeError` because both messages contain
27"undefined" destroys the signal that lets the team find each one. The criteria
28in step 1 exist to keep that from happening.
29 
30## Available tools
31 
32| Tool                                           | Purpose                                                |
33| ---------------------------------------------- | ------------------------------------------------------ |
34| `posthog:query-error-tracking-issues-list`     | Find candidate duplicate issues                        |
35| `posthog:query-error-tracking-issue`           | Pull compact details for an individual issue           |
36| `posthog:query-error-tracking-issue-events`    | Sampled `$exception` events with stack and message     |
37| `posthog:error-tracking-issues-merge-create`   | Merge existing issues into a target                    |
38| `posthog:error-tracking-issues-split-create`   | Surgically split fingerprints back out if a merge errs |
39| `posthog:error-tracking-grouping-rules-create` | Auto-group future events into one issue                |
40| `posthog:error-tracking-grouping-rules-list`   | Check existing grouping rules before adding new ones   |
41| `posthog:error-tracking-issues-partial-update` | Rename or re-describe the target after a merge         |
42 
43## Merge vs grouping rule
44 
45The two tools solve different halves of the problem:
46 
47- **Merge** is one-shot. It collapses existing issues into a target and re-attaches
48  their events. Future events still group by their original fingerprints — if the
49  same noisy pattern keeps producing new fingerprints, merging is a treadmill.
50- **Grouping rule** is durable. It rewrites the fingerprint of any matching
51  event to `custom-rule:<rule_id>` at ingestion time, so all future matches
52  share one canonical fingerprint rather than spawning new ones. The first
53  match either creates a new issue keyed off that fingerprint, or routes to
54  whatever issue is already bound to it.
55 
56Use both together when the issue is recurring: merge historical duplicates
57into a target issue, then create the rule. The rule API does **not** accept a
58target issue ID — once the rule starts firing, the resulting `custom-rule:...`
59issue can be merged into the same target so the consolidation sticks. Use
60merge alone for historical sprawl that you don't expect to recur. Use a
61grouping rule alone for a brand-new pattern you're getting ahead of, when
62you don't need to consolidate with an existing issue.
63 
64## Workflow
65 
66### Step 1 — Confirm the duplicates
67 
68Search by exception type or message to find candidates:
69 
70```json
71posthog:query-error-tracking-issues-list
72{
73  "searchQuery": "TypeError: Cannot read property",
74  "status": "active",
75  "limit": 50,
76  "orderBy": "occurrences",
77  "dateRange": { "date_from": "-30d" }
78}
79```
80 
81For each candidate, pull one sampled exception event to compare stack, type,
82and message:
83 
84```json
85posthog:query-error-tracking-issue-events
86{
87  "issueId": "<candidate_issue_id>",
88  "limit": 1,
89  "verbosity": "stack"
90}
91```
92 
93Run this once per candidate. The tool defaults to `onlyAppFrames: true`, which
94makes the top in-app frame stand out at a glance. If two candidates share the
95same top frame and same exception type, they're likely the same error — but
96verify against the full checklist below before merging.
97 
98#### Are they the same error?
99 
100Treat two issues as duplicates only when **every one** of these matches:
101 
102- `$lib` is the same SDK (`posthog-js`, `posthog-python`, `posthog-node`,
103  `posthog-android`, etc.). Errors from different SDKs almost always come from
104  different code paths even when the exception type matches.
105- The exception type is identical (`$exception_types`).
106- The top in-app stack frame points at the same file and same function. Line
107  numbers and minor offsets within that function are fine; a different file or
108  a different function on top means a different bug.
109- The message follows the same template, with differences confined to volatile
110  data — IDs, hashes, timestamps, dynamic paths. If the difference is a
111  different verb, object, or operation, it's a different bug.
112- `$exception_handled` agrees (both handled or both unhandled). A caught
113  variant and an uncaught variant are different code paths and benefit from
114  staying separate.
115 
116If any single one of those differs, they are not duplicates — investigate
117separately (`investigating-error-issue`).
118 
119#### What NOT to group together
120 
121These are the failure modes that destroy debugging signal. Do not group
122across any of them, even when the user describes them as "the same kind of
123bug":
124 
125- **Frontend and backend variants of the same exception type.** A `TypeError`
126  from a browser bundle and a `TypeError` from a Node service share a name and
127  often a message word, but the stack, the runtime, and the fix all differ.
128- **Different SDKs / platforms.** `posthog-js` vs `posthog-python` vs
129  `posthog-android` are different call sites.
130- **Same type, different file or function on top of the stack.** A
131  `NullPointerException` thrown from `OrderService.cancel` is not the same bug
132  as one thrown from `PaymentService.refund`, even if both messages say
133  "user was null".
134- **Caught vs uncaught.** Two issues that differ only in `$exception_handled`
135  are usually a code path that swallows the error in one place and lets it
136  propagate in another — keeping them separate makes that visible.
137- **Conceptually-similar bugs that happen to share a phrase.** "Cannot read
138  property of undefined" appears in many independent bugs. Without matching
139  stack frames, message similarity alone is not enough.
140 
141### Step 2 — Pick the target issue
142 
143Pick the issue that should absorb the others:
144 
145- **Most occurrences** — keeps the dominant issue so dashboards stay continuous
146- **Best name and description** — if the user has annotated one, prefer it
147- **Earliest `first_seen`** — preserves the original timeline
148 
149Note the target's ID. The other candidates become `ids` to merge in.
150 
151### Step 3 — Merge existing duplicates
152 
153```json
154posthog:error-tracking-issues-merge-create
155{
156  "id": "<target_issue_id>",
157  "ids": ["<duplicate_id_1>", "<duplicate_id_2>", "..."]
158}
159```
160 
161Merge is destructive (annotation `destructive: true`) — once issues are merged
162into a target, the source issues are gone from the active list. Confirm the
163target with the user before calling. Cap each merge call at ~50 source IDs to
164keep failures localized; for larger sprawl, batch.
165 
166Merged changes may not appear in the issue list immediately — re-listing right
167after the call can still show the source issues for a short window. If a
168follow-up `error-tracking-issues-list` call looks unchanged, wait a few seconds
169and re-query rather than re-issuing the merge.
170 
171If after the merge the target's metadata looks wrong (a duplicate had a better
172name), use `error-tracking-issues-partial-update` to fix the name or description
173on the target rather than re-merging.
174 
175### Step 4 — Decide if a grouping rule is warranted
176 
177A grouping rule is worth creating when both are true:
178 
179- The pattern keeps producing new fingerprints (you have seen new duplicates
180  appear since the last merge)
181- You can describe the pattern with property filters that won't accidentally
182  swallow unrelated errors
183 
184The canonical exception properties (`$exception_types`, `$exception_values`
185for messages, `$exception_sources` for file paths, `$exception_functions` for
186function names) are arrays at capture time. The property filter compiler
187[special-cases them](https://github.com/PostHog/posthog/blob/master/posthog/hogql/property.py#L904) — it parses the JSON-materialized column
188and wraps the filter in `arrayExists(v -> ..., JSONExtract(...))`, so all
189the standard operators (`exact`, `is_not`, `icontains`, `not_icontains`,
190`regex`, `not_regex`) work against individual elements with the bare value:
191`exact "TypeError"`, not `exact '["TypeError"]'` or `regex '"TypeError"'`.
192 
193The singular forms (`$exception_type`, `$exception_message`) and
194`$exception_stack_trace_raw` are emitted on a fraction of a percent of events;
195filtering on them produces a rule that silently never matches.
196 
197If the volatility is in the message (e.g.,
198`TypeError at /static/main.<hash>.js`), a regex filter on `$exception_values`
199works. If the volatility is in line numbers within a known file, `icontains`
200on `$exception_sources` does. `$exception_handled` is also a useful narrowing
201dimension — separate handled vs unhandled rather than mixing them.
202 
203Skip the grouping rule when:
204 
205- The duplicates are historical (one-off backfill, no new occurrences) — merge
206  is enough
207- You can't write a filter narrow enough to be safe — broaden the merge cadence
208  instead and revisit later
209 
210### Step 5 — Create the grouping rule
211 
212Translate the step 1 "same error" checklist into rule filters. A rule that
213matches more loosely than the checklist will silently merge unrelated bugs
214forever — the rule is more dangerous than the merge because it runs against
215every future event. At a minimum, scope by SDK and exception type, and add
216a third dimension (file path via `$exception_sources`, or a specific message
217phrase via `$exception_values`) to pin the call site:
218 
219```json
220posthog:error-tracking-grouping-rules-create
221{
222  "filters": {
223    "type": "AND",
224    "values": [
225      {
226        "type": "event",
227        "key": "$lib",
228        "operator": "exact",
229        "value": "posthog-js"
230      },
231      {
232        "type": "event",
233        "key": "$exception_types",
234        "operator": "exact",
235        "value": "TypeError"
236      },
237      {
238        "type": "event",
239        "key": "$exception_sources",
240        "operator": "icontains",
241        "value": "/static/checkout/"
242      },
243      {
244        "type": "event",
245        "key": "$exception_values",
246        "operator": "icontains",
247        "value": "Cannot read property"
248      }
249    ]
250  },
251  "description": "Cleanup: collapse noisy checkout TypeError fingerprints (posthog-js)"
252}
253```
254 
255Rules are evaluated in order. List existing rules first
256(`posthog:error-tracking-grouping-rules-list`) — if a rule already partially
257covers the pattern, prefer adjusting its filter over stacking a near-duplicate.
258 
259The optional `assignee` field auto-assigns issues created by the rule. Skip it
260unless the user explicitly wants ownership baked into the rule.
261 
262### Step 6 — Verify and consolidate
263 
264Sample the merged issue's recent events to confirm the merge succeeded.
265Watch for the rule's `custom-rule:<rule_id>` fingerprint to start matching
266events — the first match creates a new issue (or routes to whatever was
267already bound to that fingerprint). To keep events under your historical
268target rather than scattered across the new custom-rule issue, run a second
269merge folding the custom-rule issue into the target.
270 
271If new (non-rule) fingerprints continue appearing despite the rule, its
272filter is too narrow — widen it.
273 
274## Tips
275 
276- The user often confuses grouping rules with assignment rules. Grouping rules
277  decide _which_ issue an event lands in. Assignment rules decide _who_ owns the
278  resulting issue.
279- Don't merge issues that "look similar" without inspecting events. Two
280  `TypeError`s in different files are different bugs.
281- Stack frames are the canonical grouping signal — ingestion already
282  fingerprints on the stack, so a stable stack groups itself. A grouping rule
283  is for cases where the natural fingerprint sprays (volatile filenames,
284  hashed function names, dynamic line numbers) and you need to override it.
285- Disabling or tightening a grouping rule does not retroactively un-group
286  existing events; future events route correctly, past events stay where they
287  are. Use `error-tracking-issues-split-create` if you need to surgically
288  separate fingerprints back out of a merged issue.
289- Grouping rules are visible in the UI under Project settings → Error tracking →
290  Grouping rules; mention this when the user asks where rules live.
291