Introducing Experiments+

Timothy Chan
Wed Jun 30 2021
FEATURE-FLAGS SOFTWARE-DEVELOPMENT AB-TESTING DATA-SCIENCE ANNOUNCEMENT

At Statsig, we’ve already transformed the process of deploying new features by backing every feature gate (feature flag) by an A/B test during rollout. This provides a complete view of each feature’s performance against your company’s suite of metrics. The best part… this is all done automatically; Zero overhead, no extra steps, and this doesn’t change how long your rollout takes.

But what if you needed something even more powerful? What if you want to run A/B/n experiments? Or need to avoid experimental collision? What if you prefer a detailed report focussed on validating your hypotheses? Introducing Experiments+.

What is Experiments+?

Experiments+ is Statsig’s formal A/B/n testing and experimentation tool. It lets experimentalists:

  1. Set up hypotheses, define key metrics, and set a target completion date
  2. Set audience targeting rules
  3. Deploy multiple test groups, at custom percents
  4. Set up layers (aka Universes) allowing you to avoid collisions by running mutually exclusive experiments
  5. Receive a detailed report to evaluate your hypotheses against the key metrics.
  6. Holistically understand the experiment’s impact on your company’s entire suite of metrics. This is done with our popular “Pulse” view that helps you understand the primary, secondary and ecosystem effects. This allows you to generalize learnings, creating new hypotheses and ideas.
Sample of a Detailed Experiments+ Report

3-Step Creation Flow

At Statsig, we firmly believe A/B Testing doesn’t need to be difficult. In line with this philosophy, we’ve built a 3-step creation flow for Experiments+.

3-step Creation Flow for Experiments+

Learn More

This is just the first version of Experiments+. We have many more exciting features in the works and would love to hear from you on what else we should be building and whether this is works for you.

Want to check out Statsig? You can signup for a free account at http://www.statsig.com. You’ll be able to use our SDK and console and start building immediately. You can also play with our demo account at https://console.statsig.com/demo which includes Experiments+.


Try Statsig Today

Explore Statsig’s smart feature gates with built-in A/B tests, or create an account instantly and start optimizing your web and mobile applications. You can also schedule a live demo or chat with us to design a custom package for your business.

MORE POSTS

Recently published

My Summer as a Statsig Intern

RIA RAJAN

This summer I had the pleasure of joining Statsig as their first ever product design intern. This was my first college internship, and I was so excited to get some design experience. I had just finished my freshman year in college and was still working on...

Read more

Long-live the 95% Confidence Interval

TIMOTHY CHAN

The 95% confidence interval currently dominates online and scientific experimentation; it always has. Yet it’s validity and usefulness is often questioned. It’s called too conservative by some [1], and too permissive by others. It’s deemed arbitrary...

Read more

Realtime Product Observability with Apache Druid

JASON WANG

Statsig’s Journey with Druid This is the text version of the story that we shared at Druid Summit Seattle 2022. Every feature we build at Statsig serves a common goal — to help you better know about your product, and empower you to make good decisions for...

Read more

Quant vs. Qual

MARGARET-ANN SEGER

💡 How to decide between leaning on data vs. research when diagnosing and solving product problems Four heuristics I’ve found helpful when deciding between data vs. research to diagnose + solve a problem. Earth image credit of Moncast Drawing. As a PM, data...

Read more

The Importance of Default Values

TORE

Have you ever sent an email to the wrong person? Well I have. At work. From a generic support email address. To a group of our top customers. Facepalm. In March of 2018, I was working on the games team at Facebook. You may remember that month as a tumultuous...

Read more
ANNOUNCEMENT

CUPED on Statsig

CRAIG

Run experiments with more speed and accuracy We’re pleased to announce the rollout of CUPED for all our customers. Statsig will now automatically use CUPED to reduce variance and bias on experiments’ key metrics. This gives you access to a powerful experiment...

Read more

We use cookies to ensure you get the best experience on our website.

Privacy Policy