Product Updates

We help you ship faster. And we walk the walk
6/14/2024
Permalink ›

Advanced Local Metrics on Warehouse Native

Local Metrics are metrics that are scoped to an individual experiment. They let you create a custom metric you want to measure in the context of your experiment or gate, without having to clutter up your Metrics Catalog with these metrics. Local Metrics can be created from the Setup tab sections of your experiment. They will be calculated for the duration of your experiment or rollout, and then will cease to exist when you make a decision on your experiment.

Our April release allowed you to create simple metrics (sums, count, unique user count). By popular demand, we've unlocked all the flexibility available in the metrics catalog - windowing, SQL filters and metrics that span multiple metric sources.

image

Margaret-Ann Seger
Head of Product, Statsig
6/13/2024
Permalink ›

đź”˝ Pulse Metric Lifts View Selector

Today, we’re making it easier to view your Pulse metric lifts in the way that makes the most sense for you, whether this is viewing relative (%) deltas, absolute deltas, relative or absolute topline impact. To make this easier across all metrics in your Scorecard, we’ve introduced a selector inline in Pulse to toggle how you want to view your metric lifts.

Pulse View Selector

We’ve also introduced the ability to view your daily and cumulative time-series either by Deltas (today’s default) or Totals (raw units). This new view exists within the Pulse detail hovercard.

Deltas vs. Totals View

Check out these new views and let us know what you think!

Margaret-Ann Seger
Head of Product, Statsig

ℹ️ Metric Detail Cards

At Statsig, we believe that all builders on a team should be able to create the metrics that best measure what they and their teams care most about. This is why we built a powerful Custom Metrics product which enables you to build metrics with more complex business logic on top of raw events and input metrics.

As a team’s Metrics Catalog grows, it becomes increasingly important to provide visibility to non-metric creators on how a metric is defined to ensure they can correctly interpret metric lift results.

To this end, we’re excited to start rolling out a new Metric Detail Card. Now on hover in Pulse, you can get more information about a metric’s-

  • Definition

  • Input events

  • Applied statistical methods

  • Possible dimensions to break down by

  • …and more!

Check it out in your Pulse Scorecard and Explore queries today!

Metric Detail Card

New Suite of Javascript SDKs

We’ve rewritten our Javascript SDKs (statsig-js, statsig-react, statsig-react-native, statsig-react-native-expo, statsig-js-lite, and statsig-js-on-device-eval-client) to reduce the package sizes, codify common initialization and callback patterns, and share core logic between each package. This new SDK also supports packages for Session Replay and Web Analytics.

Read more about the migration path here, or learn about the improvements in our blog post.

Meta-analysis/Experiment Timeline View

We're rolling out the first in a series of views that enable meta-analysis across your corpus of experiments. This view lets you to filter down to experiments a team has run. At a glance you can answer questions like

  1. What experiments are running now?

  2. When are they expected to end?

  3. What % of experiments ship Control vs Test?

  4. What is the typical duration?

  5. Do experiments run for their planned duration - or much longer or shorter?

  6. Do experiments impact key business metrics - or only shallow or team level metrics?

  7. How much do they impact key business metrics?

It is rolling out now to experimenters that have expressed interest in or have offered feedback on meta-analysis needs. Reach out in Slack if this is an area of interest; we're adding more over the summer and would love to hear from you! It is homed under the Insights tab in the left navigation.

meta-analysis
5/31/2024
Permalink ›

SDK events are now hourly in WHN

If you choose to use the Statsig SDKs to log custom events (user actions like e.g. clicks, searches or page loads), we now batch and write them to your warehouse hourly (instead of daily). This is helpful when you've just launched an experiment and want to preview data to see if it is working (or causing crashes).

If you use your own telemetry pipeline to capture events, this has no impact on you.

Exposures

If you use the Statsig SDKs for assignment, we batch, dedupe and write exposure information to your warehouse daily.

When you load Pulse (experiment scorecard), we perform a just-in-time update of exposures for the current day so you're looking at current results. This typically captures exposures as recent as ~15m old.

Akin Olugbade
Product Manager, Statsig
5/24/2024
Permalink ›

Saved Queries in Metrics Explorer

We’re excited to introduce a new feature in our Product Analytics product called Saved Queries. This update improves your insight gathering and exploration workflow by providing a lightweight way to save your analysis for later use, or share them with the whole team.

What’s New with Saved Queries

Previously, to leverage previous insights you've come to in Statsig product analytics, you had to bookmark the URL or save it to a dashboard. Now, with Saved Queries, you can preserve these insights more easily. This feature simplifies how you work with data, making it more straightforward to revisit and build upon your previous analyses.

How It Works

When you land in Metrics Explorer, our main product analytics surface, you start on an “Unsaved Exploration” and can immediately dive into user and product data. If you discover valuable insights, you can save the query directly, enabling you to access and refine your findings at any future point.

Personal and Published Queries

Saved Queries come in two types:

• Personal Queries: These are private and only accessible by you. You can save any number of insights that are particularly relevant to your ongoing projects.

• Published Queries: If your saved query could benefit the entire team, you can choose to publish it. Once published, these are available to everyone in the project through the Saved Query catalog in Metrics Explorer, enabling better collaboration and knowledge sharing across your team.

We believe that Saved Queries will streamline your analysis and exploration tasks, making it easier to manage and share important insights across your team.

We will start rolling out Saved Queries today. We look forward to seeing how saving queries makes building and sharing your insights easier!

Saved Queries
Margaret-Ann Seger
Head of Product, Statsig
5/22/2024
Permalink ›

đź’» Web Analytics

Web Analytics makes it dead simple to track and watch key measures for your website.

Debuting on our suite of Javascript SDKs, Web Analytics auto-captures key user-generated events like page views, errors, page performance stats, clicks, and form submits. With each of these auto-captured events, we include key user and website page metadata, making advanced exploration easy out-of-the-box.

Once you’ve gotten up and running with Web Analytics, you can easily:

  • Explore your website’s engagement via Metrics Explorer

  • Look at your user engagement trends, via Statsig’s auto-generated suite of User Accounting metrics including DAU/ WAU/ MAU, stickiness, retention, etc.

  • Curate and share your own custom set of dashboards, applying custom filters and aggregations to your suite of auto-captured events

Read more in our docs & let us know if you have any feedback or questions!

Web Analytics Dash
Margaret-Ann Seger
Head of Product, Statsig
5/20/2024
Permalink ›

🎯 Inline Targeting Criteria

Often, you may want to experiment on a specific group of people as defined by targeting criteria. To-date to accomplish this you’ve had to define a Feature Gate with your targeting criteria and then reference this Feature Gate from the experiment.

While this is useful if your targeting criteria is relatively common (and you’ll want to reuse it again in a future experiment or rollout), this can introduce extra configs and overhead if this targeting criteria is only serving the experiment in question.

We say- extra overhead and unnecessary configs cluttering up your catalog BE GONE!

Today, we’re starting to roll out the ability to define targeting criteria inline within your experiment setup. This will be accessible via the same entry point you can select an existing Feature Gate to target your experiment against (don’t worry, that capability isn’t going anywhere).

Read more in our docs here and don’t hesitate to reach out if you have any questions!

Inline Targeting Criteria
Vineeth Madhusudanan
Product Manager, Statsig
5/14/2024
Permalink ›

Stratified Sampling (for B2B experiments)

For B2B experiments on small sample sizes (or tests where a tail-end of power users drive a large portion of an overall metric value), randomization alone doesn't cut it. Your Control and Test groups may not be well balanced if your whales end up in either group.

This new Statsig feature meaningfully reduces false positive rates and makes your results more consistent and trustworthy. It tries a 100 different randomization salts and then compares the split between groups based on a metric or classification you provide to find the best balance. In our simulations, we see around a 50% decrease in the variance of reported results.

Read more about using the feature here, or learn more about how it works here. This is now rolling out on both Statsig Cloud and Warehouse Native on Pro and Enterprise tiers.

Stratified Sampling

Loved by customers at every stage of growth

See what our users have to say about building with Statsig
OpenAI
"At OpenAI, we want to iterate as fast as possible. Statsig enables us to grow, scale, and learn efficiently. Integrating experimentation with product analytics and feature flagging has been crucial for quickly understanding and addressing our users' top priorities."
Dave Cummings
Engineering Manager, ChatGPT
SoundCloud
"We evaluated Optimizely, LaunchDarkly, Split, and Eppo, but ultimately selected Statsig due to its comprehensive end-to-end integration. We wanted a complete solution rather than a partial one, including everything from the stats engine to data ingestion."
Don Browning
SVP, Data & Platform Engineering
Recroom
"Statsig has been a game changer for how we combine product development and A/B testing. It's made it a breeze to implement experiments with complex targeting logic and feel confident that we're getting back trusted results. It's the first commercially available A/B testing tool that feels like it was built by people who really get product experimentation."
Joel Witten
Head of Data
"We knew upon seeing Statsig's user interface that it was something a lot of teams could use."
Laura Spencer
Chief of Staff
"The beauty is that Statsig allows us to both run experiments, but also track the impact of feature releases."
Evelina Achilli
Product Growth Manager
"Statsig is my most recommended product for PMs."
Erez Naveh
VP of Product
"Statsig helps us identify where we can have the most impact and quickly iterate on those areas."
John Lahr
Growth Product Manager
"The ability to easily slice test results by different dimensions has enabled Product Managers to self-serve and uncover valuable insights."
Preethi Ramani
Chief Product Officer
"We decreased our average time to decision made for A/B tests by 7 days compared to our in-house platform."
Berengere Pohr
Team Lead - Experimentation
"Statsig is a powerful tool for experimentation that helped us go from 0 to 1."
Brooks Taylor
Data Science Lead
"We've processed over a billion events in the past year and gained amazing insights about our users using Statsig's analytics."
Ahmed Muneeb
Co-founder & CTO
SoundCloud
"Leveraging experimentation with Statsig helped us reach profitability for the first time in our 16-year history."
Zachary Zaranka
Director of Product
"Statsig enabled us to test our ideas rather than rely on guesswork. This unlocked new learnings and wins for the team."
David Sepulveda
Head of Data
Brex
"Brex's mission is to help businesses move fast. Statsig is now helping our engineers move fast. It has been a game changer to automate the manual lift typical to running experiments and has helped product teams ship the right features to their users quickly."
Karandeep Anand
President
Ancestry
"We only had so many analysts. Statsig provided the necessary tools to remove the bottleneck. I know that we are able to impact our key business metrics in a positive way with Statsig. We are definitely heading in the right direction with Statsig."
Partha Sarathi
Director of Engineering
"Statsig has enabled us to quickly understand the impact of the features we ship."
Shannon Priem
Lead PM
Ancestry
"I know that we are able to impact our key business metrics in a positive way with Statsig. We are definitely heading in the right direction with Statsig."
Partha Sarathi
Director of Engineering
"Working with the Statsig team feels like we're working with a team within our own company."
Jeff To
Engineering Manager
"[Statsig] enables shipping software 10x faster, each feature can be in production from day 0 and no big bang releases are needed."
Matteo Hertel
Founder
"We use Statsig's analytics to bring rigor to the decision-making process across every team at Wizehire."
Nick Carneiro
CTO
Notion
"We've successfully launched over 600 features behind Statsig feature flags, enabling us to ship at an impressive pace with confidence."
Wendy Jiao
Staff Software Engineer
"We chose Statsig because it offers a complete solution, from basic gradual rollouts to advanced experimentation techniques."
Carlos Augusto Zorrilla
Product Analytics Lead
"We have around 25 dashboards that have been built in Statsig, with about a third being built by non-technical stakeholders."
Alessio Maffeis
Engineering Manager
"Statsig beats any other tool in the market. Experimentation serves as the gateway to gaining a deeper understanding of our customers."
Toney Wen
Co-founder & CTO
"We finally had a tool we could rely on, and which enabled us to gather data intelligently."
Michael Koch
Engineering Manager
Notion
"At Notion, we're continuously learning what our users value and want every team to run experiments to learn more. It's also critical to maintain speed as a habit. Statsig's experimentation platform enables both this speed and learning for us."
Mengying Li
Data Science Manager
Whatnot
"Excited to bring Statsig to Whatnot! We finally found a product that moves just as fast as we do and have been super impressed with how closely our teams collaborate."
Rami Khalaf
Product Engineering Manager
"We realized that Statsig was investing in the right areas that will benefit us in the long-term."
Omar Guenena
Engineering Manager
"Having a dedicated Slack channel and support was really helpful for ramping up quickly."
Michael Sheldon
Head of Data
"Statsig takes away all the pre-work of doing experiments. It's really easy to setup, also it does all the analysis."
Elaine Tiburske
Data Scientist
"We thought we didn't have the resources for an A/B testing framework, but Statsig made it achievable for a small team."
Paul Frazee
CTO
Whatnot
"With Warehouse Native, we add things on the fly, so if you mess up something during set up, there aren't any consequences."
Jared Bauman
Engineering Manager - Core ML
"In my decades of experience working with vendors, Statsig is one of the best."
Laura Spencer
Technical Program Manager
"Statsig is a one-stop shop for product, engineering, and data teams to come together."
Duncan Wang
Manager - Data Analytics & Experimentation
Whatnot
"Engineers started to realize: I can measure the magnitude of change in user behavior that happened because of something I did!"
Todd Rudak
Director, Data Science & Product Analytics
"For every feature we launch, Statsig saves us about 3-5 days of extra work."
Rafael Blay
Data Scientist
"I appreciate how easy it is to set up experiments and have all our business metrics in one place."
Paulo Mann
Senior Product Manager
We use cookies to ensure you get the best experience on our website.
Privacy Policy