Primary Metrics

Primary metrics are the main indicators that are directly tied to the specific hypothesis of an experiment. They are the immediate outcomes that you expect to change as a result of your experiment. These metrics are necessary (but not sufficient) for your experiment to succeed.

For example, if you're testing a new feature on your website, your primary metric could be the number of clicks on that feature. This is the first thing you would expect to see if your experiment is successful.

Example: Clicks, 60-second page views, conversions.

Secondary Metrics

Secondary metrics, on the other hand, are additional metrics that you monitor to ensure that you're not accidentally causing regressions or negative impacts elsewhere. These metrics help you understand the broader impact of your changes beyond the primary metrics.

For example, if you make a feature more prominent on your website, your secondary metrics could include usage on other parts of the product to ensure that the new feature isn't cannibalizing usage elsewhere.

Example: Usage on other parts of the product, click-through rates, total conversions.

Relationship Between Primary and Secondary Metrics

Primary and secondary metrics work together to give a holistic view of the impact of an experiment. While the primary metric measures the direct impact of the change, the secondary metrics ensure that the change isn't negatively impacting other areas.

In other words, the primary metric tells you if your experiment is working as expected, while the secondary metrics tell you if the experiment is causing any unintended side effects. Both are crucial for a comprehensive understanding of your experiment's results.