Out of the many high bandwidth conversations at the conference and in the weeks following the event, one stands out: Statsig founder and CEO Vijaye Raji joined another ex-Facebook builder, Sean Taylor, in a podcast conversation with hosts Tristan and Julia of dbt Labs’ The Analytics Engineering Podcast.
This episode, appropriately titled “Minimum Viable Experimentation,” covers a lot of ground in just 45 minutes. Avoiding any spoilers, the takeaways that I’m still thinking about as I type this are centered around the following main ideas:
Distributed decision-making and increased trust across teams are just a few of the business outcomes increased by experimentation that highly scalable organizations strive to achieve.
It’s hard enough to get good data scientists, so once you have a strong team, the right tooling empowers them to focus on strategic decision-making and experimentation instead of tedious (and turnover-inducing) number crunching.
Which metrics matter the most to your business will intrinsically reveal themselves
In addition to the themes above, just a few of the standout quotes from each podcast participant include:
“Build products really fast and experiment all the time.”- Vijaye
“Folks who haven’t existed in that environment…it’s a little bit hard to imagine all the things that have to be working right in order to get this [experimentation] flywheel moving.” - Tristan
“You also have to have a culture of being okay to shut down work that you’ve spent several months building and the numbers aren’t showing the numbers you’re hoping for…that’s a big cultural acceptance for a company” - Julia
“A very small number of experiments are successful. Maybe one in ten or one in six….you always learn something from an experiment; even the failed ones.” - Sean
If you’re like me and typically listen to podcasts at 1.25x or 1.5x speed, take my advice and play this conversation at normal 1x; there are so many hard-hitting points to ponder and this episode absolutely flies by. Check it out 👇
At Facebook, the company value "move fast and break things" allowed for extreme productivity and output. With the proper guardrails, it is an unbeatable way to operate.
Good release management best practices like putting code behind a software gate allows you to improve the software release cycle further.
A partial rollout is the practice of showing a feature to a percentage of users between 0% and 100% so you can see the impact a feature has on a sample of users.
Firebolt kicked off season 2 of their podcast series The Data Engineering Show with a bang by tackling questions with Statsig founder and CEO Vijaye Raji.
A/B and A/B/n testing web pages is an effective and accurate way to determine which design is most impactful.
In 2022, we dug deeper, doubling down on our core product, expanding the suite of tools we offer, and bringing more people (customers, employees, investors, and partners) on board.
Explore Statsig’s smart feature gates with built-in A/B tests, or create an account instantly and start optimizing your web and mobile applications. You can also schedule a live demo or chat with us to design a custom package for your business.