Frequently Asked Questions

Date of slack thread: 6/18/24

Anonymous: Hello, statsig team, an appeling question on long maturing metrics is here))

If I have an ingestion pipeline and one of the conversions might have quite a long time (week or two) is there a way I can count all the necessary events from leads who came about the finish date of the experiment?

The problem is coming when you stop the experiment and if you want to receive the conversions from the users who haven’t “lived” enough while the experiment was running, right now my team is using “abandon” option to stop the experiment and hence for the latest users such a long-term metric is not received and statsig reflects only the data that was already received by the time the exeriment was stopped

Do you know any workarounds for such a case? I’ve started to think about changing the traffic allocation to 0% instead of stopping the test explicitly, will it work? Would the data be supplied to the experiment for the long term metrics in this case, while the new users exposure was already stopped?

Anonymous: Hey, could you, team, please have an extra look the bot shared the outdated docs as usual, I liked the idea of baking window, I haven’t found such a chapter in the relevant docs though, so I guess to combine the window + 0% allocation to receive the metrics via ingestion pipeline from the last users should be an appropriate option?

Anonymous: Literally I’ve covered with my clicks all the metrics type, trying to create a new one and I haven’t found a setting for Baked Metrics, whereas this page from the relevant docs says that it’s possible (at the bottom)

Timothy Chan (Statsig): You can turn down allocation to 0%, which will stop new users from allocating to the experiment, while allowing the analytics to continue coming in. The only caveat is that existing users may not get the same treatment (if it’s an ongoing experience). There are ways to get around this (eg. sticky bucketing).

Baked metrics are a feature that’s built for Warehouse Native and isn’t currently supported on Cloud. If this is something you’re interested on, I can see if the feature is currently eligible for beta testing. Let me know.