Managing Path Cleaning Rules

1---
2name: managing-path-cleaning-rules
3description: 'Inspects URL paths and proposes, tests, orders, and applies project-level path cleaning rules so dynamic segments (numeric IDs, UUIDs, slugs, dates) collapse into readable aliases. Use when the user says "clean the paths", "normalize URLs", "group similar pages", "too many distinct paths", "/users/123 and /users/456 are the same page", "set up path cleaning", or asks why a Web analytics or Paths breakdown is fragmented across thousands of nearly-identical URLs. Covers regex syntax (re2), alias placeholder convention, rule ordering, the test workflow, and applying rules via the project-settings-update MCP tool.'
4---
5 
6# Managing path cleaning rules
7 
8Path cleaning rules normalize `$pathname` and `$entry_pathname` so that pages
9sharing the same template (`/users/123/profile`, `/users/456/profile`, …) collapse
10into one row (`/users/<id>/profile`) in Web analytics tiles, Paths insights, and
11any HogQL query that calls `apply_path_cleaning`. They are the right answer when
12a breakdown is fragmented across thousands of near-identical URLs.
13 
14This skill teaches you how to:
15 
16- recognize when path cleaning is the right tool
17- inspect real paths to find what needs cleaning
18- write `regex` + `alias` rules in re2 syntax with the project's placeholder
19  convention
20- test rules before saving them
21- order rules so specific patterns aren't swallowed by generic ones
22- apply the rules via MCP
23 
24## Data model
25 
26`Team.path_cleaning_filters` is a JSON list of `PathCleaningFilter` objects:
27 
28```json
29{
30  "regex": "/users/\\d+/profile",
31  "alias": "/users/<id>/profile",
32  "order": 0
33}
34```
35 
36- **`regex`** — a [re2](https://github.com/google/re2/wiki/Syntax) pattern. No
37  need to escape `/`. Anchor with `^` / `$` when you mean it.
38- **`alias`** — the literal replacement. Use angle-bracket placeholders
39  (`<id>`, `<slug>`, `<uuid>`, `<date>`) by convention so the cleaned path stays
40  human-readable. The alias is _not_ a regex template — backreferences are not
41  supported.
42- **`order`** — integer. Rules apply **sequentially** in `order` ascending,
43  each rule's output feeds the next.
44 
45Application is `replaceRegexpAll(pathname, regex, alias)` per rule, chained.
46Source: `posthog/hogql/property.py:613`.
47 
48## Workflow
49 
50### 1. Confirm path cleaning is the right move
51 
52Ask yourself: is the user complaining about cardinality (too many distinct paths
53in a chart), or do they want a per-URL drill-down? Path cleaning is for the
54former. If they want per-URL data, suggest a property filter on `$pathname`
55instead.
56 
57### 2. Inspect the real paths
58 
59Don't guess at patterns — query them. With the `execute-sql` MCP tool:
60 
61```sql
62SELECT properties.$pathname AS path, count() AS views
63FROM events
64WHERE event = '$pageview'
65  AND timestamp > now() - INTERVAL 7 DAY
66GROUP BY path
67ORDER BY views DESC
68LIMIT 200
69```
70 
71Scan the result for:
72 
73- numeric IDs: `/users/123`, `/orders/4242`
74- UUIDs: `/sessions/8f3c1a3b-…`
75- slugs: `/posts/why-i-love-posthog`
76- dates: `/archive/2024-09-12`
77- locales: `/en-US/`, `/fr-FR/`
78- pagination: `?page=3`, `/page/3/`
79 
80### 3. Draft regex + alias
81 
82| Pattern             | Example match          | `regex`                      | `alias`                |
83| ------------------- | ---------------------- | ---------------------------- | ---------------------- |
84| Numeric segment     | `/users/123/profile`   | `/users/\d+/profile`         | `/users/<id>/profile`  |
85| UUID v4             | `/sessions/8f3c1a3b-…` | `/sessions/[0-9a-f-]{36}`    | `/sessions/<uuid>`     |
86| Slug                | `/posts/why-posthog`   | `/posts/[a-z0-9-]+$`         | `/posts/<slug>`        |
87| ISO date            | `/archive/2024-09-12`  | `/archive/\d{4}-\d{2}-\d{2}` | `/archive/<date>`      |
88| Locale prefix       | `/en-US/about`         | `^/[a-z]{2}-[A-Z]{2}/`       | `/<locale>/`           |
89| Trailing query/page | `/blog?page=3`         | `\?page=\d+$`                | (empty alias drops it) |
90 
91Anchoring rules of thumb:
92 
93- start the regex with `^` only when the segment must be at the beginning of
94  the path
95- end with `$` to keep a generic rule (e.g. `\d+$`) from matching mid-path
96  segments
97 
98### 4. Test before saving
99 
100Three options, pick one:
101 
102- **Settings page tester**: `/settings/project#path_cleaning` has a built-in
103  "test path" input that replays the full ordered chain.
104- **Project HogQL** (via `execute-sql`):
105 
106  ```sql
107  SELECT replaceRegexpAll('/users/42/profile', '/users/\d+/profile', '/users/<id>/profile')
108  ```
109 
110  Chain `replaceRegexpAll` calls in the same order the rules will run if you
111  want to verify multi-rule interaction.
112 
113- **Built-in AI helper**: there is already an `AiRegexHelper` modal accessible
114  from the rule editor (`Help me with Regex` button) that turns natural
115  language into a regex. Suggest it to the user when they say "I don't know
116  regex" — but always validate the output against real paths via the tester.
117 
118### 5. Order rules from most-specific to most-general
119 
120Sequential application means a generic rule placed first will swallow
121everything that should have hit a specific rule.
122 
123```text
124order=0  /users/me/profile        →  /users/me/profile     (specific, runs first)
125order=1  /users/\d+/profile       →  /users/<id>/profile
126order=2  /users/[a-z0-9-]+        →  /users/<slug>          (catch-all, runs last)
127```
128 
129If `/users/[a-z0-9-]+` ran first it would also match `/users/me/profile` and
130make the more specific rule unreachable.
131 
132### 6. Apply via MCP
133 
134Use the `project-settings-update` tool with the full list (the field is
135replaced, not merged):
136 
137```json
138{
139  "path_cleaning_filters": [
140    { "regex": "/users/me/profile", "alias": "/users/me/profile", "order": 0 },
141    { "regex": "/users/\\d+/profile", "alias": "/users/<id>/profile", "order": 1 },
142    { "regex": "/users/[a-z0-9-]+", "alias": "/users/<slug>", "order": 2 }
143  ]
144}
145```
146 
147Always **read the existing rules first** (project settings include
148`path_cleaning_filters`) and merge — overwriting silently destroys whatever the
149team has already configured.
150 
151## Where the rules apply
152 
153When the user (or a HogQL query) opts in:
154 
155- Web analytics: the **Path cleaning** toggle in the page header
156  (`PathCleaningToggle.tsx`)
157- Paths insights: the path cleaning toggle in the insight filters
158- HogQL: any query that calls `apply_path_cleaning(path_expr, team)`
159 
160The rules are stored once per project — they are not insight-scoped.
161 
162## Common pitfalls
163 
164- **Backreferences in `alias` need double-escaping** — ClickHouse's
165  `replaceRegexpAll` supports `\0` (whole match) and `\1`–`\9` (capture
166  groups). In a JSON field or SQL string literal the backslash must be
167  doubled, so use `\\1` in `path_cleaning_filters` / HogQL to get the `\1`
168  backreference at the ClickHouse layer.
169- **Forgetting `$`** — `\d+` without an end anchor matches every numeric run
170  in any path, so `/blog/2024-09-12/post` becomes
171  `/blog/<num>-<num>-<num>/post` when you only meant to match the year
172  segment. Use `\d+$` or `\d+(/|$)` depending on intent.
173- **Escaping `/`** — re2 does not require it. `\/` works but adds noise.
174- **Case sensitivity** — re2 is case-sensitive by default. Use `(?i)` at the
175  start of the pattern for case-insensitive matching, e.g. `(?i)/users/\d+`.
176- **Replacing the whole list** — `path_cleaning_filters` is overwrite, not
177  append. Always start from the current list.
178- **Rules apply globally** — adding a rule can change historical numbers in
179  every Web analytics / Paths chart that has cleaning enabled. Warn the user
180  before applying anything destructive.
181