Platform
Resources
Product Updates
Refreshing Pulse (Experiment Scorecards)
Coming soon to Statsig is a new look to Pulse - our Experiment Scorecard page. As we added more features to this page (Target Apps, Allowed Reviewers, Hypothesis...) metric lifts got pushed off the screen. The new version will bring back focus on the metrics when the experiment is running. You can still access any of the context you're used to with one click.
The Summary tab is unchanged with this refresh. Reach out in Slack with any feedback you have - we're keen to listen as we make this view better.
đź“Ť Local Metrics on Warehouse Native
Local Metrics are metrics that are scoped to an individual experiment. They let you create the exact specific custom metric you want to measure in the context of your experiment or gate, without having to clutter up your Metrics Catalog on an ongoing basis.
Local Metrics can be created from the Setup tab sections of your entity, will be calculated for the duration of your experiment or rollout, and then will cease to exist when you make a decision on your experiment.
This rolled out on Statsig Cloud recently, and is now available on Statsig Warehouse Native too.
Schemas for Dynamic Configs
Today, we're excited to add the ability to define a schema for your dynamic configs, making it easier and less error-prone for multiple team members to collaborate together on dynamic configs.
To define a schema, you’ll see an optional “Schema” definition unit at the top of the page. When you add a schema, it will validate your rule & default return value JSON against this schema and block saving changes until the return values match the schema.
Inline Comments for Dynamic Configs
You asked and we answered! One top feature request has been the ability to add inline comments to dynamic configs to help provide clarity and visibility on field values and definitions.
With commenting, you can add comments to any new or existing dynamic config by escaping with "//". Comments will render in the dynamic config editor, but will not be sent down to your SDK.
🧮 More Metric Flexibility on WHN
Ratio Metrics: In addition to creating ratio metrics from Metric Sources, you can now create ratio metrics over existing metrics from your metric catalog.
Multi-source metrics: You can now create metrics that span multiple Metric Sources. e.g. If you have setup Online and Offline Purchases as separate metric sources, you can create a metric that sums up purchases from both metric sources.
🌸 Dashboards Spring Cleaning
We've recently made some quality of life improvements to help you find dashboards more easily and quickly derive meaningful insights.
Dashboard Summaries and Settings
When viewing dashboards, we've made it simpler to grasp the most important insights at a glance. Charts on dashboards now, by default, show the latest metric value and the percent change over time for the metric being plotted. This makes understanding current product health more straightforward than ever. You can also edit the summary value being display as the latest value, average value, or cumulative value of the metric over the time range.
Favorite Dashboards
You can now mark dashboards as favorites if they are particularly important to you. Your favorite dashboards will always appear at the top of your dashboard list, making them easier to access and ensuring you quickly get to the data that matters most.
Popular Dashboards
We've also added a new "Popular Dashboards" section to the dashboard list view. This feature makes it easy to discover dashboards that are popular within your project, helping your team share insights and context around the dashboards and product data that are proving most insightful.
📤 Templates
Have a standard rollout you want to leverage across the team? Want to standardize best practices for experiment design across the company? Templates enable you to codify a blueprint for config creation that fellow team members can use to bootstrap their own feature gates and experiments.
Key features of Templates
Create a new template from scratch from within Project Settings or easily convert an existing experiment or gate into a template from the config itself
Manage your templates all in one place within Project Settings, restricting which roles on your team have the ability to create and modify templates via Statsig's role-based access controls
Restrict which templates a given team can select from via "Allowed Templates" settings within team settings
Read more about Templates via our documentation here.
🍯Persistent Assignment on Experiments
This is a new SDK feature that makes user bucketing decisions on experiments sticky even in situations they wouldn't have been previously. Users exposed to an experiment are bucketed into Control or Test deterministically (see how); however allocation or targeting changes can cause a user to to be excluded from an experiment after they were exposed to it. Persistent Assignment ensures that users stay bucketed in the experiment even when allocation or targeting changes.
Some scenarios this unlocks
1. You can roll out an experiment to 100% of users for a week, and then drop allocation to 0%. Users exposed to the experiment in that first week will continue to experience the experiment; other users will not.
2. When you target an experiment at set of users (e.g. low engaged users), if the user state changes they usually fall out of the experiment. With Persistent Assignment they will continue to see the experiment (e.g. even if they move from low engaged -> high engaged).
Learn more about Persistent Assignment on Client SDKs, Server SDKs
🚨Alerts++
We’re excited to launch a big update to our rollout alerts product today. Here’s a quick overview of what’s changing:
Alerts will now have any experiment-level configured stats methodologies applied- If you’ve enabled CUPED or Sequential Testing for your experiment, this will now be incorporated into your alert firing logic post-the first 24 hours of a new gate/ experiment going live (i.e. as soon as daily Pulse results are available).
Alerts now have confidence intervals attached- Instead of just surfacing metric value relative to a threshold, we will surface metric value with confidence intervals attached, to provide you more context on how seriously you should be concerned about an alert firing.
Alerts only fire if they are statistically significant- This should drastically reduce the noisiness of alerts on Statsig and ensure you’re only getting pinged with high-signal regressions.
As a reminder, alerts can be set up and configured via the Metrics tab, at the per-metric level. Hop on in, set up alerts for your key regression metrics, and let us know if you have any feedback!
Support for Percentile Metrics
We're excited to announce Percentile metrics on Statsig Warehouse Native! Percentiles are often used to optimize app performance, understand feature adoption or even manage resource utilization when experimenting on backend infra and AI models.
Percentiles are particularly useful when applied to metrics that exhibit large variances. They also help understand the distribution of a metric, and can be critical to understand outliers or unusual metric behaviors. Customers can now visualize understand impact (or even alert on) p90, p95, p99, p99.9 or any other percentile.
Reach out in Slack if you want to opt into this! If you're interested in the underlying math, we'll be writing about it but it's loosely patterned on the thinking here - Applying the Delta Method in Metric Analytics: A Practical Guide with Novel Ideas.
Loved by customers at every stage of growth
