Product Updates

Reusable Cohorts

Saved and Reusable Cohorts is a powerful new feature in our product analytics at Statsig, designed to simplify tracking and understanding user behavior. This feature enables you to define, save, and reuse detailed definitions of user groups based on their interactions with your product. These definitions can range from simple behavioral patterns to complex combinations of activities or shared user properties. Cohorts are useful for recognizing different engagement phases and adjusting your strategies to better meet your users’ needs. Now, when you create an interesting or useful cohort, you can save that definition for later reuse.

Overview

Cohorts provide an important way to segment users for analysis. You can create cohorts that capture basic actions, detailed interactions, or define groups by user properties that signify different levels of engagement. Once defined, these cohorts can be easily reused by the entire team.

Examples Engagement Levels: Track users who consistently engage with specific features, such as those who have purchased an item 5 times in the last month.

Behavioral Sequences: Monitor the progression of users who follow a specific sequence of actions, like signing up, engaging with a tutorial, and using a premium feature, all within their first few days. This helps identify how well new features or tutorials convert new users into active users.

Re-engagement: Identify users who have re-engaged with a feature after a period of dormancy, such as those who performed an event in the last 7 days but had not performed the event in the 7 days prior.

Cohorts are a powerful tool for data analysis, offering valuable insights into user behavior. Saved and Reusable Cohorts make using cohorts easier than ever.

Outlier Detection on WHN

In some cases, your metrics can be materially influenced by a few outliers. This check will alert you when this is the case, so you can evaluate if using tools like Winsorization or Capping is appropriate. It will also make sure capping is not unduly influencing results if already applied.

This feature is available on Statsig Warehouse Native.

🧢 Capped Metrics on WHN

Capped Metrics are now available on Statsig Warehouse Native. With this capability, you can define max values for a metric for whatever unit type(s) are configured for this metric. Any value surpassing the set cap will automatically be adjusted downward to match it.

For instance, if you determine that purchases greater than $10,000 per day on your E-commerce platform should not skew analysis, any transaction exceeding this threshold will be adjusted downward to this limit, ensuring the integrity of your experiment analysis.

Capped metrics are available for Count and Sum metric types. Read more in our docs.

Configuring a max cap on a Custom Metric

Differential Impact Detection

Statsig can now automatically flag when sub-populations respond very differently to an experiment. This is sometimes referred as Heterogeneous Effect Detection or Segments of Interest.

Overall results for an experiment can look "normal" even when there's a bug that causes crashes only on Firefox, or when feature performs very poorly only for new users. You can now configure these "Segments of Interest" and Statsig will automatically analyze and flag experiments where we detect differential impact. You will be able to see the analysis that resulted in this flag.

Learn about how this works or see how to turn this on in docs. This feature is available on Statsig Warehouse Native (and is coming to Statsig Cloud soon).

New look and feel on Statsig

Over the Seattle summer, you will see a refreshed new look on Statsig. Building on the updated navigation, we have consolidated search, project settings and notifications into the left rail. We have simplified the gate/experiment creation screens and made 47 other little tweaks. Let us know what you think in Slack!

Statsig in 2024 is a far richer product than in 2021. We've added new product surfaces like Product Analytics and Session Replay. Experimentation has added features ranging from Meta-Analysis to Stratified Sampling. Feature flag creation lets you configure automated rollouts, ownership, and change approvers. The new look aims to keep Statsig friendly and easy to use even as we add this power. Stay tuned...

Statsig Managed API Proxy

A large but little-known problem in experimentation and feature flagging is tracking blockers (like ad blockers, VPNs, and corporate firewalls).

These can end up blocking feature flags, experiments, and even runtime dynamic config data, resulting in bugs or the exclusion of users. This has a big impact on results - biasing tests and/or reducing statistical power.

We just rolled out a new feature to solve this problem: a managed API proxy. This gives you a unique URL to send and receive data to/from Statsig servers, which makes it way less likely to be intercepted by client-side or DNS-side blockers.

It's super easy to set up (literally one click) and available to all pro tier and enterprise customers.

Read our docs for full details: https://docs.statsig.com/infrastructure/managed-proxy

A New and Modern Home Experience

We are excited to introduce a redesigned Statsig Home experience that puts your most important information at your fingertips. Our goal is to make it easier for you to access key insights about your product and your Statsig project, all in one place. The new Home experience is customizable, allowing you to display up to three widgets at a time for a personalized view of your data and activities.

Widgets:

• Recently Visited: Quickly jump back to the last places and entities you’ve visited in Statsig, making it easier to pick up where you left off without extensive searching.

• Project Updates: Keep track of important changes like Feature Rollouts or new Experiment launches. This widget can be expanded to provide a detailed view of updates in your project, including Experiment and Feature release velocity. Stay on top of what’s new and what’s changing at a glance.

• Recently Seen Users: Get a quick snapshot of new user growth over the last day, week, and month. Additionally, this widget provides a quick view into specific users recently seen in your product, which is particularly valuable for startups tracking new users onto their products.

• More coming soon!: We will be adding new widgets that you can swap into your Home experience over time. Upcoming widgets include a dedicated Velocity widget to monitor project speed, a Billing widget to track your Statsig usage, and an Ingestions widget to keep tabs on your data imports.

Dashboard:

You can now pin any existing Dashboard to your Home experience. This feature brings the most important data about your product front and center, ensuring you maintain a continuous sense of how your product, feature releases, or experiments are performing. By keeping key metrics in plain sight, you can make informed decisions faster.

The new Home experience will start rolling out today. As always, let us know what you think!

Improved Formula Support in Product Analytics

We’ve updated our formula functionality to make your product analytics explorations easier. Now, when you add a formula to a chart, all other series will be hidden by default. This helps you focus on the key results without distractions, making it simpler to analyze the specific data points you need. If you need to see the hidden series, simply click on them to unhide.

This change is especially useful when adding series to create percentages, as these values are often much smaller than the input series. Hiding the other series by default keeps your focus on the relevant percentage values.

You can also save charts or add them to dashboards with specific series hidden. This feature makes it easier to view and share the results of your formula-based explorations, keeping your visualizations clear and focused on the most relevant insights.

This feature is available now. We look forward to your feedback!

User Management v2

We're excited to introduce our new Users view, providing a central place to understand and manage users at an individual level. This update brings several powerful features to help you learn about your users, debug issues, and manage their experience.

New Features:

1. Event Log Stream:

   • View a dense, informative stream of user events in a table format.

   • Quickly drill down into individual events to see detailed logs, making it easier to understand user actions.

2. Session Stream:

   • Organize events by user sessions to get a clear picture of user interactions.

   • Easily access session replays for a visual playback of user experiences, helping you see exactly how users navigate your product.

3. Session Replays Tab:

   • View all saved recordings for a specific user in the Session Replays tab.

   • Great for diagnosing issues and understanding the reasons behind user behavior.

4. Overrides Tab:

   • Simplified management of Feature Gate and Experiment overrides for any unit ID, including individual users or entire organizations.

   • Tailor the product experience at both individual and organizational levels.

5. User Properties:

   • Get a quick snapshot of user properties, including email, country, operating system, and browser details.

   • Helps you understand your user base and their technical environment.

These new features provide broad and clear insights into user behavior, make troubleshooting easier, and simplify managing features. By viewing detailed user properties and organizing events by sessions, you can enhance profiling and better understand your user base. Start exploring the new Users view today to leverage these enhanced tools for better user understanding and management.

Meta-analysis/Metric Correlation View

We're rolling out the second in a series of new views that enable meta-analysis across your corpus of experiments. This lets you visualize and scan for correlation across metrics.

When to use it

Often the metric you want to move isn't very sensitive and takes a while to measure. It is helpful to find metrics that are more sensitive and faster to measure - and run experiments on this.

This view lets you plot two metrics on the same chart - each data point is an experiment's impact on them. You can quickly get a sense for whether the metrics tend to move together - or not. You can also remove outliers, filter down to a team's experiments or download the underlying dataset.

In this hypothetical example - "Checkouts" is the metric you want to move, but it's not very sensitive. "AddToCart" correlates well with "Checkouts", while "ViewItemDetail" doesn't.

Also see - Meta-analysis Experiment Timline View.

(Reach out in Slack if you'd like to get early access to this)