More often than not, you’ll need knowledge of SQL (or other query language), good understanding of your underlying data model and schema to get started, and the time and server compute to run often expensive, slow queries.
The Events Explorer is the newest addition to our suite of tools at Statsig and aims to simplify this process. By taking a sample of product events over a two week period that is representative of your overall product data, we allow you to perform lightning fast queries that run in seconds, not minutes with no SQL required. The Events Explorer helps you quickly and easily understand trends in your product and user data, with low lift and no technical background knowledge.
There are three main ways to use the Events Explorer that I would like to share with you today; the Table View, the Samples View and the Time Series.
To follow along and play with the examples yourself, log in to our demo account here, and click the links below each image.
The Table View allows you tobreakdown your data with aggregations. You might visit this view if you wants answers to questions such as;
How many people reach the checkout screen per day? How many of those actually complete a purchase?
How many purchases do we have per country/age-group/etc? What is the average amount spent per purchase?
What events are being logged by the product? How many of each event?
This can be a great place to start to understand what is being logged — the default view will show you the count of all events. This query quickly reveals 103M product_view events, 2.2M checkout_events and 0.8M purchase_events being logged over the last week.
From here, we are only a few clicks away from drilling down to more complex aggregations to answer questions we might have. Let’s use this view to answer the question: “Which countries spend the most on purchases and what is the average purchase price per country?” From the previous query, you simply need to (1) filter down to the purchase_event, (2) group by country and (3) aggregate the sum/average for the purchase_events to answer this question.
In only a few clicks, we have an answer: the bulk of purchase_event value is driven by the US and India ($33M and $17M), where-as Mexico and Canada drive much less by comparison. The average purchase price is quite similar across different countries.
We currently support Count, Sum and Average aggregations, but plan to support more powerful aggregations in the future.
The Time Series allows you to explore all of your aggregations and groupings across the axis of time. Take our previous query looking at purchases in the US and India; plotting it as a time series allows us to quickly see which hours different regions are active, and can reveal trends in usage.
Monitoring product usage trends can be a powerful tool to help you prioritize which features you should be working on. It can help you confirm the work you’re doing is impactful to your topline and is aligned with the way people use your product.
This view is also very powerful operationally, as you release new products and monitor for product errors.
You can monitor the gradual rollout of a product or feature, i.e. watching usage/impressions/clicks start to ramp up in a specific region or for a new feature.
You can watch for spikes in error logs, understand which users were affected and when, and monitor rolling out a fix as errors return to baseline.
You can quickly gain context on spikes in app errors or reported issues; i.e. Are these errors specific to a certain region? A certain platform? Are they affecting iOS, Android, Web, or all three? Did the time these errors started spiking coincide with any gate or experiment launches?
If a user or employee reports an issue, you can quickly confirm if this issue is widespread or localized i.e. is there a sharp drop in checkout impressions for everyone or only on our staging tier.
The time series view has settings to view data at different time granularities (ranging from 5min to 1day), allows for overlaid comparisons i.e. the week/week comparison for a single event, and can show up to 8 series in one chart.
Finally, the Samples View allows you to see sample rows for events that you care about. This is a great starting place to explore data when you may be unfamiliar with the underlying data schema or what exactly is being logged. With this you can quickly see exactly which fields are being logged, what data is being logged in each field and what kind of filters or groupings you might be able to apply to each field.
One way we use this view at Statsig is to audit and understand what we log per event to inform which logging and telemetry we are missing/should consider adding in future sprints.
Another common usecase for the samples view is to view detailed events for a single user, to reconstruct their user journey, which can be useful to understand for new user activations or for users reporting product issues.
Overall, the Events Explorer aims to provide fast, easy to use product observability, regardless of your familiarity with technical query languages or the underlying design of the data model.
I’m fortunate to work with some super talented colleagues on this, and we’d all love to hear any feedback you might have while trying out the product. Please feel free to share any feedback you have in our slack channel here, and tag myself, Ritvik, Anu or Jason.
Thanks to our support team, our customers can feel like Statsig is a part of their org and not just a software vendor. We want our customers to know that we're here for them.
Migrating experimentation platforms is a chance to cleanse tech debt, streamline workflows, define ownership, promote democratization of testing, educate teams, and more.
Calculating the right sample size means balancing the level of precision desired, the anticipated effect size, the statistical power of the experiment, and more.
The term 'recency bias' has been all over the statistics and data analysis world, stealthily skewing our interpretation of patterns and trends.
A lot has changed in the past year. New hires, new products, and a new office (or two!) GB Lee tells the tale alongside pictures and illustrations:
A deep dive into CUPED: Why it was invented, how it works, and how to use CUPED to run experiments faster and with less bias.