🚨 Alerts++

🚨Alerts++

We’re excited to launch a big update to our rollout alerts product today. Here’s a quick overview of what’s changing:

• Alerts will now have any experiment-level configured stats methodologies applied- If you’ve enabled CUPED or Sequential Testing for your experiment, this will now be incorporated into your alert firing logic post-the first 24 hours of a new gate/ experiment going live (i.e. as soon as daily Pulse results are available).

• Alerts now have confidence intervals attached- Instead of just surfacing metric value relative to a threshold, we will surface metric value with confidence intervals attached, to provide you more context on how seriously you should be concerned about an alert firing.

• Alerts only fire if they are statistically significant- This should drastically reduce the noisiness of alerts on Statsig and ensure you’re only getting pinged with high-signal regressions.

As a reminder, alerts can be set up and configured via the Metrics tab, at the per-metric level. Hop on in, set up alerts for your key regression metrics, and let us know if you have any feedback!

Alert configuration UX