Experimentation best practice dictates that an experiment should have a highly targeted set of metrics that you’re actively trying to move, along with a broader swath of metrics you’re monitoring to ensure you don’t regress.
Today, we’re adapting our Scorecard to reflect this best practice, and putting in place some smart limits on the Scorecard—max 10 Primary Metrics and 40 Secondary Metrics. Coming soon will be the ability for Enterprise customers to specify an even tighter limit on Scorecard metrics via their Org Settings if desired.
One bonus implication of these limits is that we’re auto-expanding tagged metric groups, making it even easier to see (and manage) all the individual metrics being added to your Scorecard when you add a metric tag.
Let us know if you have any feedback or questions on this change!
We've just started rolling out Experiment Policy controls to customers with Enterprise contracts. Configure good defaults for experiment settings like Bayesian vs Frequentist, Confidence Intervals (or optionally even enforce them). Find it under Organization Settings ➜ Settings ➜ Experiment Settings