debugging-experiments
Debug and support PostHog Experiments (A/B tests) for a customer looking at their own results. Use whenever an experiment support ticket is pasted or a customer asks a results question, most commonly "why aren't my exposures even?", "why is one variant getting no traffic?", "why am I missing / seeing too few exposures?", "why does the bias banner show?", or "why don't PostHog's numbers match my SQL?". Pulls the experiment's real data read-only, matches it to a known-cause catalog, and produces a customer-facing explanation, fix, and review of the pertinent numbers. Loads diagnosing-experiment-results as its deep diagnostic library. DO NOT TRIGGER when: creating an experiment (use creating-experiments), only configuring rollout (configuring-experiment-rollout) or metrics (configuring-experiment-analytics), asking lifecycle questions (managing-experiment-lifecycle), or the underlying feature flag is what's misbehaving rather than the results (use debugging-feature-flags).
Pinned to revision 4e8964e24a5a, so it is the text this page describes rather than whatever the author pushed since.
Files
- skills/debugging-experiments/SKILL.md
- skills/debugging-experiments/references/customer-reply.md
- skills/debugging-experiments/references/pulling-the-data.md
- skills/debugging-experiments/references/real-vs-noise.md
- skills/debugging-experiments/scripts/srm_check.py
Every link opens the file at its source, pinned to the revision this page describes.