When Chander joined Motion in late February 2022, there were three critical problems facing the engineering team:
Incredibly long and unstable release processes
No ability to roll back releases
Chander recognized all three issues were stemming from a lack of feature flagging infrastructure.
Like any new Head of Engineering whose team was drowning, Chander decided to buy (vs. build)—especially since feature flags were not their core product. And while he was correct in his diagnosis, his takeaway from the whole experience was "I was horribly incorrect in choosing LaunchDarkly."
LaunchDarkly had been recommended by several peers Chander had worked with at previous startups, and he admittedly didn’t put much thought into platform evaluation. "Everything was horribly broken, and we needed to move fast, so (I thought to myself) let’s just do it."
By April 1st LaunchDarkly was mandatory for every new feature in Motion’s backend, web app, and Chrome extension. "By July 1st it was obvious this was a huge mistake" Chander states. The team began evaluating alternatives and began integrating Statsig into their codebase in August.
By November 1st LaunchDarkly was completely phased out.
Statsig’s pricing model is usage-based, whereas "LaunchDarkly stubbornly insists on seat-based pricing, so every engineer we added had an additional cost"—as Chander points out.
Motion tried to restrict access to a select few, but it added extra burden on those few engineers to keep track of everyone’s launches and feature flags. This included things like reminding them to turn flags on after a release, cleaning them up after a launch, and adding new ones to the test environment.
Ultimately it became too laborious, and the team decided the developer productivity cost wasn’t worth it. They also ran into some issues with how LaunchDarkly was estimating MAUs—something that raised questions and could have caused substantial overcharges.
Motion ran into performance and memory issues using integrating LaunchDarkly in a Chrome extension using the documentation provided. Chander ultimately found a workaround by using the LaunchDarkly JS Client SDK on Github and adapting it for the team’s needs. He states "while it mostly works, I’m 100% sure there are some underlying bugs. I’m using code in a way it wasn’t meant to be used, so please don’t blindly copy paste this code wherever."
We share Chander’s belief that a good feature flagging system should make it as easy to remove a feature flag as it does to add a new feature flag. According to him "LaunchDarkly fails spectacularly yet again in this regard."
The team at Motion found it impossible to tell if a feature flag was actually being used or not. Additionally, they found the insights graph inside LaunchDarkly was also not accurately representing flag evaluation volumes after several updates were made.
Motion ran into many more challenges with LaunchDarkly, including:
No support for Expo in React Native, which meant their mobile apps couldn’t use feature flags. (Note: Statsig supports Expo.)
Inconsistent updates that impacted some users and resulted in getting the wrong values (this all went away after switching to Statsig.)
Issues with mistakenly archiving flags between test and prod environments due to UI/UX design.
Running into an upcharge fee in order to mandate approvals on feature flags (Statsig provides this for free).
Virtual hugs to you Chander. This whole experience sounds frustrating and we're sorry you had to go through it.
It’s critically important to evaluate vendors and digital infrastructure providers before spending hours of valuable engineering talent implementing (and potentially unwinding/replacing) a solution.
Statsig loves learning from customer experiences. Often, we’re in a position to help fix and clean up what was obviously a poor fit: It's why we have an engineering team dedicated to customer success, and why our product roadmap is prioritized to solve customer needs (we ship new features every two weeks!).
Plus, 75% of Statsig customers use us for feature flagging in addition to our experimentation and analytics capabilities.
Kong is our Typescript-based write-once-run on every SDK framework. “Write once, run anywhere” is always a dream for programmers, and now we have just that!
LaunchDarkly was mandatory for every new feature in Motion’s backend, web app, and Chrome extension. "It was obvious this was a huge mistake."
Last Tuesday, Statsig brought a cadre of data science and experimentation fans together at a loft space in San Francisco for the first-ever Data Science Meetup.
Well-designed experimentation is the first step in creating a rollout structure that consistently delivers optimal results—whatever they may be.
Using data and experimentation, the Obama 2012 campaign generated over one billion dollars in donations, nearly $700,000,000 of which were online.
It’s only my first week yet, but each day I am more and more impressed by the team’s velocity, excitement, and transparency, and feeling more sure that I’ve made the right decision for /me/.
Explore Statsig’s smart feature gates with built-in A/B tests, or create an account instantly and start optimizing your web and mobile applications. You can also schedule a live demo or chat with us to design a custom package for your business.