Download this in PDF format: Whatnot Case Study

The Problem

“Testing new features in production was incredibly difficult & time-consuming.”

That’s how Rami Khalaf, Product Engineering Manager at Whatnot, described rolling out new features at the organization. His engineering team was bogged down with managing divergent release cycles for each app platform, rolling out new builds, and waiting for users to upgrade before learning how the new releases were performing.

“Not only did slow release-cycles get in the way of delivering value to users, they also took invaluable time from our small, single digit engineering team”, Khalaf explained.

At Whatnot, enabling anyone to turn their passion into a business is the mission. They’re building the future of ecommerce; an interactive community where creators can go-live to safely buy, sell, and geek out with collectors and other like-minded people.

The primary reason for the engineering bottleneck, according to Khalaf, was a function of Whatnot’s real-time, ephemeral livestreams. Sellers start streaming whenever they want, list items for sale and can stop streaming at any time. As a result, testing new features that relied on live interaction was incredibly difficult & time-consuming.

Khalaf needed a solution that enabled his team to quickly and confidently test the impact of new releases, in a real production environment, before rolling out to their entire population of users.

The Solution

Internal Dogfooding with Statsig Feature Flags

To solve these challenges & speed up release cycles, Khalaf crafted an “Internal Dogfooding” framework at Whatnot. Using Statsig Feature Flags, his team can now quickly ship every rollout into a real production environment, strictly to internal Whatnot employees only, to test how it lands.

Everyone on the Whatnot team dogfoods the new release to quickly identify major issues & gather initial feedback. By using Statsig to gate new features to internal Whatnot employees only, in a real production workload, Khalaf & his team can confidently test and polish features before they roll out to real users.

Canary Releases with Statsig Feature Flags

Internal Dogfooding is one thing, but to ensure total confidence in new rollouts, Khalaf also needs to gather feedback from real users. To do this safely, his team uses Statsig to enable new features to a very small subset of all public users- say, 5% of their total population (their “canaries”).

Using Statsig Metrics , Whatnot engineers can immediately see the impact that their new release has on Whatnot’s key business metrics, accelerating their time from building → learning → iterating.

“One feature we launched did great in Internal Dogfooding, but as soon as we enabled the same feature for a small portion of our beta users, we quickly identified a significant uptick in request timeouts for all users”, Khalaf explained.

Using Statsig Feature Flags, Khalaf can quickly uncover issues like these, roll back the feature with ease, and deploy a fix within hours. Whatnot repeats this process for new releases until the team is confident that each & every new feature is ready to be rolled out to all public users.