Feature gates (aka feature flags) are a game-changer for building software. I wrote recently on my personal LinkedIn about how building this way during my time at Facebook was a fundamental shift in my own experience as an engineer.
Yet, as Uncle Ben says, with great power comes great responsibility.
Every powerful tool has tradeoffs, and extensively making use of feature flags can quickly lead to a complicated codebase. Each time you branch the code in your app, server or website, you have to maintain parallel paths at the same time and make sure everything works correctly in both.
After a while of building and releasing in this powerful way, your codebase can start to make your head spin.
The antidote to this problem is to consistently go back and remove gates that you no longer need. Once you finish testing your feature and decide if the right move is to launch it or remove it, you can go ahead and delete the code that you no longer need, and remove the check to your gate.
This sounds simple enough, but sometimes you want to keep a gate around for a while after launch, just in case you missed something and you need to disable it. Or, perhaps you’re keeping a holdout to measure impact across many gates. You might leave that in place for months at a time. How on earth do you remember to go back and find all of these code paths you no longer need?
Take our “sandbox” project for a spin! In this project, we have generated real-time sample data for you to explore and experiment with.
We’ve launched a way to differentiate between permanent gates and temporary gates, so that gates that have gone stale can automatically get marked as such.
Occasionally, you’ll need to have a feature gate stick around indefinitely. This might happen because your gate acts as a kill switch to disable functionality in an emergency. Or perhaps it’s used to control legacy functionality that only exists in older app versions that some number of users never upgrade from.
Whatever the reason, you can now mark gates as permanent so that they we can move them out of your way and so that you won’t be reminded to remove them if they appear stale (more on that below).
A feature gate is now considered stale when it meets any of the following criteria:
it’s been rolled out to everyone (or rolled back!)
it isn’t being checked in production
it hasn’t had any rules configured for some time
Your team will be able to filter for these stale gates in the feature gates catalog:
We’ll also show you suggestions for how to resolve stale gates within Statsig, such as archiving a gate or making it permanent:
Lastly, once a month, we’ll notify you (via email and/or our Slack notifications integration), which gates may be considered stale that you should consider taking action on!
Whether you’re a product manager or engineering manager looking to bring cleanliness to your feature gates, or a team member trying to bring clarity to the group, we recommend incorporating stale gates into your team rituals!
Weekly check-in: Once a week, a single team member could filter all feature gates to Stale and look for gates that might need more follow-up. Or, review together in a team meeting!
Quarterly cleanup party: Once a quarter, get the whole engineering and product team in a room to search together for stale feature gates and remove unnecessary ones as quickly as you can. Try to rank highest on the leaderboard!
Connect with our data scientists and engineers, and ask questions or just hang out with other cool folks that believe in an experimentation culture!
Thanks to our support team, our customers can feel like Statsig is a part of their org and not just a software vendor. We want our customers to know that we're here for them.
Migrating experimentation platforms is a chance to cleanse tech debt, streamline workflows, define ownership, promote democratization of testing, educate teams, and more.
Calculating the right sample size means balancing the level of precision desired, the anticipated effect size, the statistical power of the experiment, and more.
The term 'recency bias' has been all over the statistics and data analysis world, stealthily skewing our interpretation of patterns and trends.
A lot has changed in the past year. New hires, new products, and a new office (or two!) GB Lee tells the tale alongside pictures and illustrations:
A deep dive into CUPED: Why it was invented, how it works, and how to use CUPED to run experiments faster and with less bias.