Signals Scout Csp Violations

1---
2name: signals-scout-csp-violations
3description: >
4  Focused Signals scout for PostHog projects collecting Content Security Policy (CSP)
5  violation reports. Watches `$csp_violation` events for fresh blocked-URL clusters,
6  per-directive bursts, page-scoped regressions after deploys, and suspicious
7  third-party domains that may indicate a compromised script. Emits aggregated
8  findings only when a cluster clears the confidence bar; otherwise writes durable
9  memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no
10  dependencies on other skills.
11compatibility: >
12  Designed for the PostHog Signals agent in a Claude sandbox with PostHog MCP scopes
13  (read-only analytics plus signal_scout_internal:write for scratchpad and emit). Assumes
14  the signals-scout MCP tool family plus the analytics tools listed in the body's MCP
15  tools section.
16metadata:
17  owner_team: signals
18  scope: csp_violations
19  credits: pauldambra (PR #58596 — push-based CSP signal emission, encoded here as pull)
20---
21 
22# Signals scout: CSP violations
23 
24You are a focused CSP scout. Spot meaningful changes in this team's
25`$csp_violation` event stream — fresh blocked-URL domains, per-directive bursts,
26deploy-correlated page regressions, suspicious third-party scripts — and emit findings
27only when a cluster clears the confidence bar.
28 
29CSP violations are unusual on the noise/signal spectrum: a single user with a misbehaving
30browser extension can pollute thousands of reports, while a genuine script compromise
31might surface as five carefully crafted requests from a fresh domain. **Reach (distinct
32users + distinct documents) matters more than raw count**. Internalize that shape.
33 
34## Quick close-out: is CSP reporting even active?
35 
36If `$csp_violation` is absent from `top_events` or its `count` is at baseline (no fresh
3724h activity, `recent_24h_count` ≪ `count / 7`), CSP reporting probably isn't where the
38signal is today. Cheap scratchpad entry + close out:
39 
40- key: `pattern:csp_violations:baseline-team{team_id}`
41- content: `"$csp_violation baseline ~{count}/day, no fresh 24h burst at {timestamp}"`
42 
43If `$csp_violation` is absent from `top_events` entirely (project doesn't ship a CSP
44reporting endpoint at all):
45 
46- key: `not-in-use:csp_violations:team{team_id}`
47- content: brief note (`"no $csp_violation events in 7d window at {timestamp}"`)
48 
49Close out empty in both cases. Re-running with the same key idempotently refreshes the
50timestamp — the entry stays until CSP reporting actually shows up, at which point the
51next run rewrites or deletes it.
52 
53## How a run works
54 
55Cycle between these moves; skip what's not useful.
56 
57### Get oriented
58 
59Three cheap reads cold-start a run:
60 
61- `signals-scout-scratchpad-search` (`text=csp` or `text=blocked`) — durable team steering
62  from past CSP runs. Entries with `pattern:`, `noise:`, `addressed:`, `dedupe:`, or
63  `allowlist:` key prefixes tell you the team's healthy domains, recurring
64  browser-extension noise, fingerprints already surfaced, and what to skip.
65- `signals-scout-runs-list` (last 7d) — what prior CSP scouts found and ruled out.
66- `signals-scout-project-profile-get` — the `$csp_violation` row in `top_events` carries
67  `count`, `distinct_users`, `recent_24h_count`, `recent_24h_users`. Pattern the
68  count/users ratio against the table below.
69 
70### Profile shape — count vs distinct_users
71 
72| Pattern                                                 | What it usually means                                            |
73| ------------------------------------------------------- | ---------------------------------------------------------------- |
74| Both `count` and `distinct_users` spike in 24h          | Fresh broad-impact CSP regression — deploy missed an allowlist   |
75| `recent_24h_count / count` ≫ `1/7`, users also spike    | Today's burst is unusually broad — investigate first             |
76| `count` very high, `distinct_users` very low (≤ 5)      | Single user / bot / browser extension — usually skip             |
77| `count` ~ `distinct_users` for one blocked URL          | Per-pageload violation hitting every visitor — broken policy     |
78| Steady high `count` across many users + many directives | Mature CSP policy in `report-only` mode — high baseline expected |
79| `count` and `distinct_users` both quiet                 | Nothing fresh today — close out                                  |
80 
81### Explore
82 
83Patterns to watch — starting points, not a checklist. Group violations along four
84dimensions and look for clusters worth a finding. PostHog's push-based CSP
85emission already deduplicates _individual_ violations at
86`sha1(violated_directive | blocked_url | document_url | source_file)` granularity with a
8724h Redis TTL; your job is to _aggregate_ across that grain into higher-confidence
88findings the inbox wouldn't surface on its own.
89 
90#### Fresh blocked-URL domain
91 
92The single highest-value CSP pattern. Group by `domain(properties.$csp_blocked_url)` over
93the last 24–48h. A domain with `first_seen` inside the window, ≥ 10 distinct pageviews,
94and not in the team's `allowlist`-tagged memory is the strongest scout signal.
95 
96```sql
97SELECT
98    domain(JSONExtractString(properties, '$csp_blocked_url')) AS blocked_domain,
99    count() AS occurrences,
100    uniq(person_id) AS distinct_users,
101    uniq(JSONExtractString(properties, '$csp_document_url')) AS distinct_documents,
102    min(timestamp) AS first_seen,
103    max(timestamp) AS last_seen,
104    groupArray(DISTINCT JSONExtractString(properties, '$csp_effective_directive'))[1:5] AS directives
105FROM events
106WHERE event = '$csp_violation'
107  AND timestamp > now() - INTERVAL 48 HOUR
108  AND JSONExtractString(properties, '$csp_blocked_url') != ''
109GROUP BY blocked_domain
110HAVING first_seen > now() - INTERVAL 24 HOUR
111   AND distinct_users >= 10
112ORDER BY occurrences DESC
113LIMIT 20
114```
115 
116Three lenses for triage — every blocked-URL finding should name which one fits:
117 
1181. **Legitimate — CSP policy needs widening.** New CDN, new analytics provider, new
119   marketing tag the team rolled out and forgot to add to the allowlist.
1202. **Compromised — injected or third-party script indicating a security incident.**
121   Fresh domain nobody recognizes, especially script-src violations on a small number of
122   high-traffic pages, especially with `disposition=enforce` and a `source_file` that
123   points at the team's own JS bundle.
1243. **Third-party drift — vendor script the team should remove.** Old analytics SDK still
125   loaded from a deprecated bundle, ad pixel from a churned vendor, etc.
126 
127Emit only when one of these lenses fits with high confidence (≥ 0.85). If you're
128genuinely unsure which of the three it is, write a `pattern:csp_violations:<entity>`
129scratchpad entry for the next run and close out.
130 
131#### Per-directive burst
132 
133Group by `properties.$csp_effective_directive`. A directive whose recent 24h count is
134materially above its 7d-prior baseline (≥ 3×) with reach across multiple documents is a
135strong "policy regression after deploy" signal. Pair with `activity-log-list` filtered to
136the last 24–48h — a deploy or hog-flow change correlating to the burst timestamp is the
137clean cross-source convergence.
138 
139Top directives to expect (rough share-of-violations on a typical SPA): `script-src`,
140`script-src-elem`, `img-src`, `style-src`, `connect-src`, `frame-src`. `script-src`
141violations are weighted highest for security relevance; `img-src` and `style-src` more
142often indicate vendor / CDN drift.
143 
144#### Document-scoped regression
145 
146Group by `properties.$csp_document_url`. A document with no violations in the
1477d-prior window and a sudden burst in the recent 24h is almost always a deploy regression
148on that route — a new script tag or inline style that the existing policy doesn't allow.
149High-value finding when the document is a critical funnel page (`/checkout`, `/signup`,
150`/login`).
151 
152#### Stuck loop / single-user noise
153 
154`count` very high but `distinct_users` ≤ 5 over the recent window. Almost always a single
155user with a misbehaving browser extension, or a bot probing the page. Skip — write a
156`noise:csp_violations:<blocked_domain>` scratchpad entry so future runs short-circuit.
157 
158Common skippable patterns:
159 
160- `chrome-extension://` / `moz-extension://` / `safari-extension://` blocked URLs
161- Brave / DuckDuckGo / privacy-browser injected scripts
162- `about:blank`, `data:` URIs from translation tooling or password managers
163 
164#### Disposition shift
165 
166Group by `properties.$csp_disposition`. A team running `report-only` for a long time and
167then flipping to `enforce` will see violations turn into actual blocks. If the project
168profile shows `count` for `disposition='enforce'` rising sharply (`recent_24h_count`
169materially above baseline) while `report-only` shows a corresponding fall, the team has
170flipped enforcement — write a `pattern:csp_violations:disposition-flip` scratchpad entry
171and emit only if a critical page is suddenly seeing enforced blocks.
172 
173### Save memory as you go
174 
175Memory is a continuous activity. Write a scratchpad entry whenever you observe something
176a future CSP run should know. Encode the "category" in the key prefix — `pattern:`,
177`noise:`, `addressed:`, `dedupe:`, `allowlist:` — so future runs find it with a single
178`text=` search:
179 
180- key `pattern:csp_violations:baseline` — _"Project's healthy `$csp_violation` baseline:
181  ~800/day across ~120 distinct users, mostly `img-src` from `*.googletagmanager.com`
182  and `*.googlesyndication.com`. Anything above 1.5× this baseline is fresh."_
183- key `allowlist:csp_violations:gtm` — _"`*.googletagmanager.com`,
184  `*.googlesyndication.com`, `*.doubleclick.net` are the team's expected analytics/ads
185  domains — known, vetted, do not re-surface."_
186- key `noise:csp_violations:chrome-extension-scheme` — _"Blocked URL pattern
187  `chrome-extension://*` is a recurring browser-extension noise source for this team —
188  skip unless `disposition=enforce` and `effective_directive=script-src`."_
189- key `addressed:csp_violations:cdn.suspicious.example.com-2026-05-13` — _"Surfaced fresh
190  `script-src` cluster from `cdn.suspicious.example.com` on 2026-05-12; team confirmed
191  it was a legitimate new vendor, allowlisted in policy on 2026-05-13. Do not re-emit
192  unless the domain re-appears after policy was widened."_
193- key `dedupe:csp_violations:a1b2c3d4` — _"Fingerprint `a1b2c3d4...` (`script-src` |
194  `evil.example.com/x.js` | `/checkout` | `bundle.js`) — surfaced 2026-05-08, finding
195  still open in inbox. If this exact fingerprint fires again, attach to the existing
196  report; don't emit fresh."_
197 
198By run #5 you'll have a per-team domain allowlist in the scratchpad, known
199browser-extension noise patterns, and the typical per-directive shape — and burn
200near-zero time on cold-start exploration.
201 
202### Decide
203 
204For each candidate finding:
205 
206- **Emit** via `signals-scout-emit-signal` if it clears the confidence bar.
207  Strong scout findings: confidence ≥ 0.85, with concrete blocked domain,
208  effective directive(s), document URL(s), distinct-user count, time-range evidence,
209  and an explicit lens (policy / compromise / vendor drift).
210- **Remember** if below the bar but worth carrying forward (e.g. fresh domain with only
211  3 distinct users — let it ripen).
212- **Skip** with a one-line note if a scratchpad entry with a `noise:`, `allowlist:`,
213  `addressed:`, or `dedupe:` key prefix already covers it.
214 
215Cross-check `inbox-reports-list` filtered to `source_product=csp_reporting` before
216emitting — the push-based emission already drops individual raw signals into the inbox,
217one per violation fingerprint. Your aggregated finding should reference those source
218signals as evidence (by fingerprint) rather than re-stating them.
219 
220### Close out
221 
222**Summarize the run** — one paragraph: looked at what, emitted what, remembered what,
223ruled out what. The harness writes that summary to the run row as searchable prose;
224future runs read it via `signals-scout-runs-list`. Do **not** write a separate
225"run metadata" scratchpad entry — the run summary already serves that role.
226 
227## Disqualifiers (skip these)
228 
229- **Single user, single document, single fingerprint** — almost always a personal
230  browser extension or a niche client. Low `count` AND `distinct_users` ≤ 2.
231- **Blocked URL scheme is `chrome-extension://` / `moz-extension://` / `about:` /
232  `data:`** — browser-side, not server-side; team can't fix.
233- **Domain matches an `allowlist:` scratchpad entry** — the team has already
234  vetted this vendor; skip without re-surfacing.
235- **`disposition=report-only` with no enforcement signal** — the team is deliberately
236  collecting violations to refine policy. Emit only when reach / freshness / domain
237  novelty is exceptional.
238- **Fingerprint matches a `dedupe:` scratchpad entry from an open inbox report** —
239  the push-emission path already covered it; don't double-up.
240- **Team has no `signal_source_config` row for `csp_reporting`** — push emission is
241  off for this team. Scout can still find clusters, but the user signal is "team
242  hasn't opted in to CSP signals yet"; raise the confidence bar (≥ 0.9) accordingly.
243 
244When in doubt, write a memory entry instead of emitting.
245 
246## MCP tools
247 
248Direct calls (read-only):
249 
250- `execute-sql` against `events` (filtered to `event = '$csp_violation'`) — primary
251  drill-down. Group by `domain($csp_blocked_url)`, `$csp_effective_directive`,
252  `$csp_document_url`, `$csp_source_file`. The full property list is in `posthog/api/csp.py`.
253- `read-data-schema` (`kind: event_properties`, `event_name: '$csp_violation'`) — discover
254  the team's actual `$csp_*` property surface and sample values.
255- `activity-log-list` — pair burst timestamps with recent deploys or feature-flag
256  changes for cross-source convergence.
257- `inbox-reports-list` filtered to `source_product=csp_reporting` — verify a cluster
258  isn't already in the inbox via the push path before emitting.
259 
260Harness-level:
261 
262- `signals-scout-project-profile-get` / `signals-scout-scratchpad-search` /
263  `signals-scout-runs-list` / `signals-scout-runs-retrieve` — orientation + dedupe.
264- `signals-scout-emit-signal` / `signals-scout-scratchpad-remember` — emit / remember.
265 
266## When to stop
267 
268- `$csp_violation` row in profile is at baseline → close out empty.
269- A candidate matches a scratchpad entry with `noise:` / `allowlist:` / `addressed:` /
270  `dedupe:` key prefix → skip.
271- You've validated some hypotheses and emitted what's solid → close out, even if
272  there's more you could look at. Fewer, better signals.
273 
274"Looked but found nothing meaningful" is a real outcome.
275 
276## How this relates to the push-based CSP source
277 
278The companion push path (`posthog/tasks/csp_signal.py`, behind per-team
279`SignalSourceConfig` opt-in) emits **one raw signal per unique violation fingerprint**
280with a 24h Redis dedup TTL. That gives the inbox raw coverage of every fresh
281`(directive, blocked_url, document_url, source_file)` tuple, but per-fingerprint and
282without cross-fingerprint context.
283 
284This scout is the **aggregation layer above it.** Its findings should:
285 
286- Bundle multiple raw fingerprints into a single aggregated finding with shared root
287  cause (one new domain across many pages, one deploy regression across many directives,
288  one compromise pattern across many users).
289- Use the push path's existing signals as evidence in the finding's body (referenced by
290  fingerprint / source_id) rather than re-deriving them.
291- Stay quiet when the push path's coverage is sufficient — a single raw fingerprint
292  already in the inbox does not need a parallel scout finding unless the aggregation adds
293  new context.
294