HelloFresh, a global digital-native CPG company and the world’s leading meal kit provider, based in Berlin, faced bottlenecks in its experimentation capabilities as they scaled globally and across teams inside the organization.

Aurora Moreno Herrera, Senior Product Owner on the Experimentation Team, recognized that their previous setup with an in-house platform was clunky and hindered their velocity.

With an 8-person experimentation team supporting rapidly growing needs across the business, HelloFresh saw an opportunity to accelerate progress with an off-the-shelf platform.

“Our goal was to enhance the quality of experiments and shorten decision-making times. The demands of our stakeholders outpaced what we could deliver,” said Aurora.

“Adopting Statsig’s off-the-shelf features felt like ‘buying time’ — it allowed us to move faster and meet our goals.”

An e-commerce business with unique challenges

HelloFresh’s business model as a direct-to-consumer meal delivery service means that measurement isn’t limited to digital interactions. “You could receive a box (meal kit) without ever needing to interact with the website,” explained Aurora.

“Any platform that only looks at interaction events wouldn’t work for us. Most of our business and product metrics are in the warehouse.” This meant HelloFresh needed a solution that could work directly on their existing datasets.

“Using multiple tools is always inefficient,” Aurora added. She highlighted two key challenges with their old tooling:

• Statistical data was only available for primary metrics
• Analysis had to be done per country, and aggregating data was a hassle

Choosing Statsig Warehouse Native

HelloFresh went through an intense evaluation process, comparing Statsig and other third-party vendors, and ultimately chose Statsig for its end-to-end capabilities and ability to meet their unique needs.

“Statsig was very responsive to our requirements. The support and collaboration during the evaluation gave us confidence that it was the right platform for us,” said Aurora.

HelloFresh’s experimentation approach categorizes three types of experiments: conversion, reactivation, and retention. Given the complexity of scenarios like weekly subscriptions, Aurora emphasized the need for running thoughtful and adaptive experiments.

“Statsig is more flexible, allowing us to easily define user allocation and data collection periods. During our evaluation, they even shipped new features that let us customize experiments to fit our business needs.”

Aurora Moreno Herrera

Senior Product Owner, HelloFresh

Furthermore, Aurora noted that Statsig’s UI was more professional than the internal one they had and gave internal stakeholders a sense of trust.

Higher quality and velocity of experiments

HelloFresh quickly began running concurrent experiments aimed at increasing conversion rates, boosting average order value, enhancing customer loyalty, and reducing error rates, among others. Now, the team has scaled to running nearly 1,000 experiments per year and increased the number of experiments with a decision by 55%, demonstrating their commitment to data-driven product growth.

Aurora noticed immediate improvements in implementation reliability, data quality, and trustworthiness. “Manual analysis has reduced thanks to Statsig. Now we get aggregated information inline with the hypothesis,” she explained.

With data-driven insights from experiments, teams are able to pay closer attention and identify problems early, ultimately improving the quality of experiments. “The log stream in the diagnostics tab makes people click through, understand events better, see what’s happening, and raise more questions,” she added.

Scaling a robust experimentation process

Aurora highlighted how Statsig is helping them standardize experiment setup and enforce best practices.

“We’re enhancing experimentation as we scale by creating templates that can be reused across teams. We’re also leveraging verified metrics as a quality guardrail, ensuring we only analyze metrics that have been validated and deemed trustworthy across the organization.”

With Statsig, HelloFresh has access to more techniques that enhance statistical rigor and have resulted in a 25% reduction in decision-making time. Aurora noted a few key features:

• CUPED
• Bonferroni correction and Benajamini-Hochberg procedure
• Sequential testing
• Pre-experiment bias detection

Partnering with the Statsig team and the road ahead

“The collaboration with Statsig has been very fruitful. Their team is highly responsive, and we’ve received valuable input around statistics and the SDK integration that has significantly improved our experimentation program.”

Aurora Moreno Herrera

Senior Product Owner, HelloFresh

The team is excited to continue scaling with Statsig. “There is a lot of potential to take advantage of all the new features that Statsig is shipping and continue improving the quality of our experiments in the months ahead.”