Great growth leaders go beyond a superficial strategy of “measuring and testing.” They need to be competent in at least two out of three skills:
A. Defining sound business models that capture the core levers of how your business grows
B. Executing rigorous and thoughtful testing and experimentation programs
C. Building sustained value that solves real problems
Let’s chalk out the frameworks to build and apply these skills. We’ll also include examples and tips to see them in action.
To build a growth engine, the most important skill is recognizing the levers that drive the business. For example, we can break down Amazon’s revenue drivers as follows.
Revenue = customers x purchases x frequency
Breaking this down further,
customers = visitors x conversion to purchase
purchases = products x quantity x price
products = categories x items per category
frequency = f (repeat purchase behavior)
To grow, Amazon must do one or more of the following:
Turns out that the top of the funnel is the least important lever for the business. This is because if you deliver on the promise to solve real problems for your customers early and often, the funnel will automatically fill itself. You just have to remove friction where it exists.
Effective growth leaders know where to experiment, which part of the funnel to focus on, what’s the right test to run vs. one that’s not interesting. Here are a couple of tips to get started on a framework.
While large companies like small wins, small companies enjoy lots of low hanging fruit. Amazon and Facebook would love to grow 1% every week because these small wins add up quickly with their large numbers. When you don’t have as many users, you should be more dramatic about the changes you aim for. For instance, to validate a 20% improvement over control, your test can complete within 7 days with just 1000 users assuming a 5% baseline conversion rate.
You have a large sample size with low base rate at the top of the funnel, and a small sample size with a high base rate at the bottom of the funnel.
Raising the base conversion rate at the top of the funnel by 10% takes it from, say 5% to 5.5%. This experiment would need a large sample size and would need to run for longer to achieve statistical significance.
Raising the base conversion rate by 10% at the bottom of the funnel takes it from, say 20% to 22%. This experiment would need a much smaller sample size and would complete within a couple of days.
With basic funnel math, it’s easy to see why you should run experiments deeper in the funnel. Especially with a product that requires a significant purchase decision towards the end, you want to run tests where customers enter their credit card details rather than the home page. A committed customer willing to pay is also more likely to respond to a change in design or experience.
Customer acquisition is a function of channel, targeting, content, and conversion.
If there’s one thing you take away today, it’s that you cannot organically grow something that sucks. This is why your growth lead needs to be a product person. Most meaningful changes that you experiment with come from a deep understanding of your customers.
Retention is a key metric to test if the product solves a real problem for customers. If you need a number to validate your product-market fit, it’s retention. There are generally three kinds of retention curves: flat, declining, increasing.
Retention can increase, for example, (i) with network effects such as on social platforms, (ii) with new products, categories, geographies, (iii) when the customer grows their frequency of usage, or (iv) customer expands consumption as more individuals in the family/team switch to using the product. Figure out what the retention curve for your business should look like.
More tactically, don’t let churn overwhelm new customer acquisition by letting your retention slide. Growth is a function of new users acquired, churned users, and resurrected users. Just tracking churned users will ensure that you’re not losing money faster than you can walk to the bank. Understand why they’re churning and which channels and information can help resurrect them.
Magic moments are when your customers “get” it, they understand what your product is about. It’s the moment they discover that it does something that they’ve never seen before. On Facebook, it’s when you see a friend. On Uber, it’s when you step out of the car at your destination with zero hassle.
You can do this the non-scalable way: ask your early customers why they use the product. You might even have an instinctive sense of what gets someone’s eyes glued to the product.
You can also do this the scalable way: find correlations. The number of items a customer purchases on Amazon is likely correlated with how often they return, or perhaps that they’re a prospective Prime member.
The company may target a strategic objective centered around an output or a north star metric. At the highest level, it could be revenue. More specifically, it could be number of hosts for Airbnb, number of posts for Twitter, number of orders for DoorDash, viewing time for YouTube, consumption of compute credits for Snowflake, and so on.
Such north star metrics are challenging to move. For example, moving year-over-year revenue growth from 25% to 26% is a big deal, but often occurs +0.1% at a time. The challenge with such north star objectives is that while the inputs are what we can control, the objectives (outputs) are what really matter. The task at every level of the organization is to decode the relationship between inputs and company-wide objectives. Measure teams against goals that track inputs and let the objectives take care of themselves.
This is the final one. Eventually you will run out of intuition and you will need data to identify blockers and experiment with new ideas. With the right metrics, data gives you empathy for your users to put yourself in their shoes.
Data also helps you make decisions faster. If everyone knows what metric to optimize for, they don’t have to kick the ball to their data science team or wait for two weeks to get on the leader’s calendar to make the decision to ship.
Data gives you a glimpse of the future. When an A/B test can show you how a feature will perform, use it! See how Statsig can serve up an A/B test for free along with a feature gate.
Got more tips? Join us on the Statsig Slack channel!
Explore Statsig’s smart feature gates with built-in A/B tests, or create an account instantly and start optimizing your web and mobile applications. You can also schedule a live demo or chat with us to design a custom package for your business.
This summer I had the pleasure of joining Statsig as their first ever product design intern. This was my first college internship, and I was so excited to get some design experience. I had just finished my freshman year in college and was still working on...
The 95% confidence interval currently dominates online and scientific experimentation; it always has. Yet it’s validity and usefulness is often questioned. It’s called too conservative by some , and too permissive by others. It’s deemed arbitrary...
Statsig’s Journey with Druid This is the text version of the story that we shared at Druid Summit Seattle 2022. Every feature we build at Statsig serves a common goal — to help you better know about your product, and empower you to make good decisions for...
💡 How to decide between leaning on data vs. research when diagnosing and solving product problems Four heuristics I’ve found helpful when deciding between data vs. research to diagnose + solve a problem. Earth image credit of Moncast Drawing. As a PM, data...
Have you ever sent an email to the wrong person? Well I have. At work. From a generic support email address. To a group of our top customers. Facepalm. In March of 2018, I was working on the games team at Facebook. You may remember that month as a tumultuous...
Run experiments with more speed and accuracy We’re pleased to announce the rollout of CUPED for all our customers. Statsig will now automatically use CUPED to reduce variance and bias on experiments’ key metrics. This gives you access to a powerful experiment...