Content Experimentation Best Practices

1---
2name: content-experimentation-best-practices
3description: A/B testing and content experimentation methodology for data-driven content optimization. Use when implementing experiments, analyzing results, or building experimentation infrastructure.
4license: MIT
5metadata:
6  author: sanity
7  version: "1.0.0"
8---
9 
10# Content Experimentation Best Practices
11 
12Principles and patterns for running effective content experiments to improve conversion rates, engagement, and user experience.
13 
14## When to Apply
15 
16Reference these guidelines when:
17- Setting up A/B or multivariate testing infrastructure
18- Designing experiments for content changes
19- Analyzing and interpreting test results
20- Building CMS integrations for experimentation
21- Deciding what to test and how
22 
23## Core Concepts
24 
25### A/B Testing
26Comparing two variants (A vs B) to determine which performs better.
27 
28### Multivariate Testing
29Testing multiple variables simultaneously to find optimal combinations.
30 
31### Statistical Significance
32The confidence level that results aren't due to random chance.
33 
34### Experimentation Culture
35Making decisions based on data rather than opinions (HiPPO avoidance).
36 
37## Resources
38 
39See `resources/` for detailed guidance:
40- `resources/experiment-design.md` — Hypothesis framework, metrics, sample size, and what to test
41- `resources/statistical-foundations.md` — p-values, confidence intervals, power analysis, Bayesian methods
42- `resources/cms-integration.md` — CMS-managed variants, field-level variants, external platforms
43- `resources/common-pitfalls.md` — 17 common mistakes across statistics, design, execution, and interpretation
44