KNOWLEDGE BASE

Useful treasure trove of knowledge

Aniket Alshi
Tuesday, July 19, 2022, 10:58 PM
Is the experiment checklist customizable? Can we add additional steps that may be specific to our project or teams?
Kevin Ye
Tuesday, July 19, 2022, 10:27 PM
Adding `Confirmation page shown` as the last metric in `test funnel metric`
Jiakan Wang (Statsig)
Tuesday, July 19, 2022, 10:00 PM
Yeah agreed. I can’t come up with a timeline right now because the “aggregated stats for diagnostics” is something that I just thought about today in our conversation, so I will need to bring it to our team to prioritize among all the other things the team is building, but it is not on our immediate roadmap.
Jacob Hurwitz
Tuesday, July 19, 2022, 9:16 PM
Ultimately the questions I want to answer are: • what percentage of our users are failing to initialize statsig? • who are those users? My inclination was that it would be easier/faster for us to get that data by logging it ourselves, but if we’re the only customer asking for this and it’s not already on your roadmap, very fair of you to push back. Based on your response, if you’re already planning on building additional capabilities into the Statsig product, that might work for us as well. I don’t think the log stream will get us these answers, since that only shows very recent exposures, and doesn’t let us drill back into the past or get aggregated stats. But “features to show aggregated stats on things like you mentioned, how often a user is using fresh value vs. cached and etc” sounds very much like this would answer our questions. Do you have a rough sense of the timeline for these features? Historically the Statsig team has impressed me with how fast feature requests have been built :slightly_smiling_face: but even just a sense of whether this is on your immediate roadmap or longer-term roadmap would help me determine whether we want to wait for these stats to be built, or if it’s worth investigating an alternative (like forking the SDK) in the meantime.
Jiakan Wang (Statsig)
Tuesday, July 19, 2022, 8:38 PM
In general providing more debugging info like this is a direction we are going with the product, e.g. we are surfacing evaluation reason (cache, localoverrides, etc) in the log stream with the new SDK you are now using, which wasn’t available in the old version, with it there is also “evaluation time”, which indicates how fresh the value is (the timestamp will be old if it’s using cache, and it’s available in the details page of log stream, which we will also make available in the users tab soon). I think going forward we will implement features to show aggregated stats on things like you mentioned, how often a user is using fresh value vs. cached and etc. I don’t think it’s realistic for us to commit to providing what you are asking for in the SDK anytime soon, so if that’s the route you want to try quickly now, I think it’s actually easier for you to fork the SDK repo and make whatever change you want to try there in the forked version. The only thing I will note is that you will probably want to change these https://github.com/statsig-io/react-sdk/blob/bff44dae8c202748267b6179af1f2fbfbb71597d/src/StatsigProvider.tsx#L142-L143|metadata in your forked version, so that it will be easier for us to debug if something’s wrong.
Eli Spitzer
Tuesday, July 19, 2022, 8:01 PM
Thanks
Vineeth
Tuesday, July 19, 2022, 6:36 PM
Braze doesn't have a native integration with us, but we have several ways to export variant assignment (webhook, CDB integrations like Segment, Rudderstack, mParticle etc, and an API to export). Is there a CDB you currently use?
Jacob Hurwitz
Tuesday, July 19, 2022, 5:47 PM
Thinking about this some more, I’d be flexible based on whatever is easiest to implement in the SDK, but my ideal might be a way to tell whether initialize timed out or succeeded every time we created a `StatsigProvider`, so we could more easily track the denominator here of all times Statsig is loaded. (If we only get failures, we’d have to approximate the denominator using other metrics.)
Ritvik (Statsig)
Tuesday, July 19, 2022, 5:39 PM
No problem! That's definitely something we want to do in Events Explorer - we just haven't prioritized working on it yet. We are very open to suggestions/feature requests here as we figure out what to build so let us know what you want! And yes, being better than Scuba and ODS combined is the goal :slightly_smiling_face:
Ken Cheng
Tuesday, July 19, 2022, 5:37 PM
This is helpful, thank you! Will Statsig be looking into supporting more ad hoc transformations on the Event Explorer? This would make Events Explorer more like a superset of Scuba + ODS in facebook terms :joy:
Jacob Hurwitz
Tuesday, July 19, 2022, 5:37 PM
One thing that I think would be helpful for us is if we’re able to log on our end how often this issue is occurring. I don’t think we can do so with the current Statsig SDK. • My first thought was to add a gate that’s supposed to always pass for everyone and log in our app whenever this gate resolves to false/fail. This will catch whenever a new client is unable to call initialize, but won’t detect when a client uses stale/cached data from days or weeks prior because it successfully called initialize, but is unable to do so now. • Looking through `StatsigClient.ts` in source code, it seems to me like there is no way to tell if `fetchValues` resolves due to the initialize succeeding, or the timeout. • It does look like `save` in `StatsigStore.ts` saves the current timestamp into the cache, so it’s possible we could try some sort of logging based on this timestamp being too old? But there seem to be cases where it’s intentionally old. To my understanding, if a user doesn’t refresh the page, we will not initialize again, and they will (intentionally) keep using the gate values they were using when they first loaded the page. Would it be possible for you to somehow augment the SDK to give us an API we could use to log on our end when initialize fails? It would be sufficient for us to log if and only if the original call plus all 3 retries all fail.
Ritvik (Statsig)
Tuesday, July 19, 2022, 5:35 PM
You cannot currently do this automatically in events explorer (you would need to do two separate queries, one for B, one for A, and then calculate the ratio). However, you can do this with custom ratio metrics! Here are the docs, let me know if you have any questions: https://docs.statsig.com/metrics/create#4-ratio-metrics
Ken Cheng
Tuesday, July 19, 2022, 5:29 PM
Question: Reading the documentation of the Events Explorer, I was wondering what the best way is to calculate the ratio of 2 events? For example, I have event A for some process being started and event B for the process succeeding. How can I best monitor B/A?
Ritvik (Statsig)
Tuesday, July 19, 2022, 5:35 PM
You cannot currently do this automatically in events explorer (you would need to do two separate queries, one for B, one for A, and then calculate the ratio). However, you can do this with custom ratio metrics! Here are the docs, let me know if you have any questions: https://docs.statsig.com/metrics/create#4-ratio-metrics
Ken Cheng
Tuesday, July 19, 2022, 5:37 PM
This is helpful, thank you! Will Statsig be looking into supporting more ad hoc transformations on the Event Explorer? This would make Events Explorer more like a superset of Scuba + ODS in facebook terms :joy:
Ritvik (Statsig)
Tuesday, July 19, 2022, 5:39 PM
No problem! That's definitely something we want to do in Events Explorer - we just haven't prioritized working on it yet. We are very open to suggestions/feature requests here as we figure out what to build so let us know what you want! And yes, being better than Scuba and ODS combined is the goal :slightly_smiling_face:
Alex Boquist
Tuesday, July 19, 2022, 5:13 PM
sweet, thanks
Ritvik (Statsig)
Tuesday, July 19, 2022, 5:09 PM
Thanks for reporting - we just caught this earlier this morning and are pushing a fix, you can use https://staging.console.statsig.com for now since the fix is there, or go to the regular console URL and the fix should be there in an hour or so
Alex Boquist
Tuesday, July 19, 2022, 5:07 PM
Error when trying to add more tests to the “Group” section on an Experiment that is not started yet
Alex Boquist
Tuesday, July 19, 2022, 5:06 PM
Alex Boquist
Tuesday, July 19, 2022, 5:06 PM
hi, when saving updates to an experiment, getting this error
Ritvik (Statsig)
Tuesday, July 19, 2022, 5:09 PM
Thanks for reporting - we just caught this earlier this morning and are pushing a fix, you can use https://staging.console.statsig.com for now since the fix is there, or go to the regular console URL and the fix should be there in an hour or so
Alex Boquist
Tuesday, July 19, 2022, 5:13 PM
sweet, thanks
tore (statsig)
Tuesday, July 19, 2022, 3:35 PM
I think Juan Diego De Miguel Mateo might have done this or something similar. But what do you mean exactly? There is a js bundle hosted on this cdn which you can use: https://www.jsdelivr.com/package/npm/statsig-js
tore (statsig)
Tuesday, July 19, 2022, 3:28 PM
Jorge Maroto Garcia
Tuesday, July 19, 2022, 3:22 PM
did you have any advance on this one Eli Spitzer?
Matt Williams
Tuesday, July 19, 2022, 2:32 PM
Maggie (Statsig) are you able to help with this one?
Alvaro Mattos
Tuesday, July 19, 2022, 2:17 PM
Thanks Maggie
Maggie (Statsig)
Tuesday, July 19, 2022, 2:11 PM
Not necessarily. When you log the event for Add Payment Details, you can include a metadata field that's `hours since account creation`, for example. Then create a custom metric that's the user count of Add Payment Details with the filter `hours since account creation < 24` Another option is to create a ratio metric `Add Payment Details / Create Account`. This will only count users that have both events in the same calendar day. So not exactly 24 hours.
Alvaro Mattos
Tuesday, July 19, 2022, 2:01 PM
So I would need to log two events: “Create Account” and “AddPaymentDetails” with timestamp?
Maggie (Statsig)
Tuesday, July 19, 2022, 1:52 PM
When you log the event for Add Payment Details, can you include time since account creation in the metadata? If so, then you can create a filtered user count metric
Alvaro Mattos
Tuesday, July 19, 2022, 1:47 PM
*Question:* Hi there. I’m working on an experiment where I want to measure how many users complete an action (Add Payment Details) within 24 hours after creating an account. What would be the recommended setup and custom events?
Maggie (Statsig)
Tuesday, July 19, 2022, 1:52 PM
When you log the event for Add Payment Details, can you include time since account creation in the metadata? If so, then you can create a filtered user count metric
Alvaro Mattos
Tuesday, July 19, 2022, 2:01 PM
So I would need to log two events: “Create Account” and “AddPaymentDetails” with timestamp?
Maggie (Statsig)
Tuesday, July 19, 2022, 2:11 PM
Not necessarily. When you log the event for Add Payment Details, you can include a metadata field that's `hours since account creation`, for example. Then create a custom metric that's the user count of Add Payment Details with the filter `hours since account creation < 24` Another option is to create a ratio metric `Add Payment Details / Create Account`. This will only count users that have both events in the same calendar day. So not exactly 24 hours.
Alvaro Mattos
Tuesday, July 19, 2022, 2:17 PM
Thanks Maggie
Omar Guenena (Lime)
Tuesday, July 19, 2022, 1:45 PM
Myroslav Sypa ^
Omar Guenena (Lime)
Tuesday, July 19, 2022, 1:44 PM
anu (statsig) FYI one of our team members tried it and got “*Response Status: Failed to fetch (CORS or Network Issue)*“, but it worked when he changed the url from ```curl --request GET 'https://api.statsig.net/console/v1/experiments' --header 'STATSIG-API-KEY: console-xxxxXXXXxxxxXXXXxxxx' ``` to ```curl --request GET 'https://api.statsig.com/console/v1/experiments' --header 'STATSIG-API-KEY: console-xxxxXXXXxxxxXXXXxxxx'```
Matt Williams
Tuesday, July 19, 2022, 9:51 AM
(just for tracking metrics)
Matt Williams
Tuesday, July 19, 2022, 9:51 AM
Hi Is it possible to implement the client SDK within Google Tag Manager?
Matt Williams
Tuesday, July 19, 2022, 2:32 PM
Maggie (Statsig) are you able to help with this one?
tore (statsig)
Tuesday, July 19, 2022, 3:35 PM
I think Juan Diego De Miguel Mateo might have done this or something similar. But what do you mean exactly? There is a js bundle hosted on this cdn which you can use: https://www.jsdelivr.com/package/npm/statsig-js
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:58 PM
No problem!
Roger Lin
Monday, July 18, 2022, 11:51 PM
Sure. I can do that. Thanks!
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:42 PM
I see. There isn’t any other downside to call `shutdown` when it’s not initialized other than the exception getting raised. We should probably remove that at some point in the SDK. For now can you catch the error around `shutdown` ?
Roger Lin
Monday, July 18, 2022, 11:33 PM
The problem is if, something went wrong in the initialization stage, then both the init stage and the desconstruct stage will raise exceptions. The latter one will complains about calling shutdown failed since statsig hasn’t been initialized.
Roger Lin
Monday, July 18, 2022, 11:32 PM
Yes. The context is that I initialize statsig once the Spring bean container is up and just call shutdown once when the bean container is down.
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:28 PM
Hi Roger Lin - usually you don’t need to check for that when you call `shutdown`, and it’s only recommended to call this once per app session when it’s about to exit. Can you share some context on why you need to check if it’s been initialized? And are you looking to call `shutdown` multiple times in the app’s lifecycle?
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:27 PM
Hi Xiaoyu Yin - if the gate does not exist (typo for example), we think the best thing to do is to return `false` , so that your app continues to function correctly. It’s recommended that during development, you check the “log stream” under the diagnostics section for the gate you are using. If it’s set up correctly, you should see logs coming in near real time to verify that everything is correct.
Roger Lin
Monday, July 18, 2022, 10:51 PM
Hello folks, I am currently using the Java SDK for integrating STATSIG into my Spring Boot application. I am wondering is there a way for me to check whether statsig has been initialized before calling the shutdown method in a ContextClosedEvent listener? Thanks for any help in advance.
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:28 PM
Hi Roger Lin - usually you don’t need to check for that when you call `shutdown`, and it’s only recommended to call this once per app session when it’s about to exit. Can you share some context on why you need to check if it’s been initialized? And are you looking to call `shutdown` multiple times in the app’s lifecycle?
Roger Lin
Monday, July 18, 2022, 11:32 PM
Yes. The context is that I initialize statsig once the Spring bean container is up and just call shutdown once when the bean container is down.
Roger Lin
Monday, July 18, 2022, 11:33 PM
The problem is if, something went wrong in the initialization stage, then both the init stage and the desconstruct stage will raise exceptions. The latter one will complains about calling shutdown failed since statsig hasn’t been initialized.
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:42 PM
I see. There isn’t any other downside to call `shutdown` when it’s not initialized other than the exception getting raised. We should probably remove that at some point in the SDK. For now can you catch the error around `shutdown` ?
Roger Lin
Monday, July 18, 2022, 11:51 PM
Sure. I can do that. Thanks!
Jiakan Wang (Statsig)
Monday, July 18, 2022, 11:58 PM
No problem!

We use cookies to ensure you get the best experience on our website.

Privacy Policy