Product Updates

Custom Experiment Checklist

We're thrilled to announce the launch of Custom Experiment Checklist, a new feature that empowers admins to tailor the experimentation guidelines to their company's specific needs. This feature allows you to replace the default Statsig experiment checklist with your own custom checklist that follows internal best experimentation practices.

Benefits:

• Ensure adherence to company-specific best practices

• Foster a unified experimentation culture across your organization

• Increase the quality and consistency of experiments

Why we built this:

Over the past few months, our customers expressed a desire for more flexibility in configuring experiment guidelines within Statsig. We listened, and Custom Experiment Checklist is our response to this valuable feedback. Custom Experiment Checklist is now available for all users. To get started, navigate to your Organizational settings and look for the new "Experiment Checklist" option.

We're excited to see how this feature will enhance your experimentation process. As always, we welcome your feedback and suggestions for further improvements.

Announcing Warehouse Native Product Analytics

We're excited to start rolling out our Product Analytics suite to Statsig Warehouse Native.

You can see the exact step in a 5-step checkout workflow where half of your users are dropping off. You can filter and slice metrics down by any property, instantly. Your Growth teams can spelunk in data and generate hypotheses to try.

All of this - with centralized data governance in your warehouse - a single source of truth, with do data duplication or drift.

This plays well with experimentation - letting you group events or a metric by experiment groups, or even look at a few sample rows of data when investigating an issue.

Metrics Explorer is in broad beta on Statsig WHN and is free for the rest of the 2024. It has been Generally Available on Statsig Cloud since March this year.

Bot Filtering

Statsig will begin filtering out known bot traffic from all exposures data in the Statsig console. These web-crawling bots can sometimes inflate exposure counts but don’t represent the users you’re most often trying to measure. This should improve the accuracy of your experiments, feature gate analytics, and user tracking. Several Statsig customers have requested this feature and we’re excited it’s finally coming.

For more information about this feature, you can check out our updated exposures docs page. Bot filtering will be turned on by default for all Statsig projects, but we’re also building an opt-out setting, which will be available in console.

Parameter Stores: Decouple your mobile app from Statsig configuration

Parameter Stores allow you to think in terms of parameters - things in your app that you want to be configurable remotely. Parameters decouple your code from the configuration in the Statsig Console. It is a level of indirection that allows you to run any set of experiments, or change gating or values on the fly, all without hardcoding an experiment name in your app. Each of these parameters that you define and check can be remapped at will between Statsig entities. Create a boolean parameter to enable a feature, turn it into a Feature Gate for internal testing, then run an Experiment when your app is released, and then turn it back into a Feature Gate that ships the winning variant to the right audience - all without a code change, or a new mobile app release.

Benjamini-Hochberg procedure

This is a statistical method that reduces the probability of false positives by adjusting the significance level for multiple comparisons. It is not as extreme as a Bonferroni Correction, because instead of controlling the chance of at least one false positive (Family Wise Error Rate), this controls the expected value of false positives when the null hypothesis has been rejected (False Discovery Rate).

This is now rolling out on both Statsig Cloud and Statsig Warehouse Native.

Learn more

First-time Filters in Funnels

We’re excited to introduce the ability to scope events and steps in funnels to the first time they were performed by users. This new feature lets you focus on analyzing the initial interactions users have with your product, allowing you to understand and enhance their first impression.

With the First-Time Filters feature, you can easily isolate and examine the first-time experience distinct from general user flows. This enables you to gain insights into how new users engage with your service and identify opportunities for improvement.

How to Use:

To add a first-time filter in a funnel, click the “…” next to the event you want to analyze and select “Filter to First Time.”

Start exploring your users’ first impressions today and unlock new opportunities for improving conversion!

Session Streams

We’re excited to rollout out Session Stream support in our Metric Drilldown and Funnel charts. Session Streams allow to contextualize a data point by giving granular, event-by-event understanding of a users’s experience before and after the data point in question.

To use Session Steams in Metric Drilldown charts, click on a data point and select “View Session Streams". This will present a list of users that performed the event in question. For each user, you can then see all of the user’s events in the date range of the data point, with the relevant event highlighted. You can also easily hide irrelevant or noisy events.

To use Session Streams in Funnels, click on a funnel step and choose "View Session Streams”. The behavior will be similar to the above with the added context of each relevant event in the funnel will be highlighted.

Session Streams will begin rolling out today!

Multi-Event Steps in Funnels

We’re introducing Multi-Event Funnel Steps feature in our product analytics suite, designed to give you greater flexibility in analyzing user behaviors. This tool allows you to group multiple, related user actions—like viewing a product, adding to a cart, or checking product details—into a single funnel step.

This functionality recognizes that different user actions can often represent similar steps in the user journey. For instance, adding an item to a wishlist and adding it to a cart could indicate a user’s interest in a product. By grouping these actions together, you can analyze them as a single step in your funnel, simplifying and adding flexibility your analysis - providing a clearer view of how users move through your product.

This approach helps you better understand the variety of ways users interact with your product, and allowing you to capture these nuances in Statsig, making it easier turn the variety of real world ways users interact with your product into useful insights.

Give Multi-Event Funnel Steps a try today!

Keyboard Navigation Improvements

We’ve made it easier to navigate Statsig with a keyboard. You can now use our search functionality (cmd+k) to navigate to specific product entry-points such as Feature Gates, Experiments, of Metrics Explorer. This makes it easier and faster to get around Statsig, especially if you prefer navigating through our interface with a keyboard instead of a mouse or trackpad. You can use the search shortcut wherever you are in Statsig. You can also use it to navigate to specific entities such as a specific dashboard, dynamic config, or feature gate.

Happy navigating!

Outlier Detection on WHN

In some cases, your metrics can be materially influenced by a few outliers. This check will alert you when this is the case, so you can evaluate if using tools like Winsorization or Capping is appropriate. It will also make sure capping is not unduly influencing results if already applied.

This feature is available on Statsig Warehouse Native.