Frequently Asked Questions

Date of slack thread: 8/12/24

Anonymous: So I’ve been trying to merge the buckets / want to make sure parts of my Next.js app for both client and server components stay in the same experiment (ie in sync, both should be in test or control). However, it seems like there isn’t an easy way to get a “source of truth” for this. I’ve managed to generate a universal “Stable ID” for helping with this but is there a way to ensure my experiment values stay the same as well?

tore (Statsig): Having a single identifier that is consistent across the front end and back end is the solution to this - statsig experiment group evaluation is based on that identifier and the group size/rollout%. Have you tried out your identifier with checks on the server and client to verify you get the same group?

Anonymous: The groups end up being different but fairly certain the identifier is the same. When I log my stable id in either location they match, and also show up in the statsig log. eg. I’m supposed to be in the Control group according to the Exposure Stream, but my server log (despite showing the same stable id, which is what I then set my userID as with useExperiment) ends up logging that it is in a “Test” group.

Anonymous: In fact now that I’m looking at this it seems like my server experiment is constantly bucketing me as “Test”, even as I use incognito mode and rerun new versions with new stable ids.

Anonymous: This is in local dev, is there a reason this would be locked in this state?

tore (Statsig): Is your experiment based on that ID? Which experiment are you checking?

Anonymous: What do you mean by based on that ID? My experiment is a 50/50 split, but since I’m using client + server Next.js code my understanding is I need to set a stable id (which then gets passed as a userID to server components).

Anonymous: More than happy to share any code / project id or whatever you need to get more info.

Anonymous: This is all in local dev by the way, no clue if that matters.

tore (Statsig): Every statsig entity has an ID type. So the gate/experiment you are checking has an ID type - either stableID, userID, or a customID. When you check it on the client side, it will evaluate using that id from the user object. Ditto on server. Right now, it sounds like you pass the identifier around, and sometimes its considered the user id, and sometimes its the stableid.

Anonymous: Oh, I think my assumption was to use the stable ID as the userID in the server object like so:

const useExperiment = await getExperimentSync(
    {
      userID: stableId ?? "",
    },
    "feed-as-homepage"
  );

Is this incorrect syntax if my plan is to use stable id server side?

tore (Statsig): Ah, I see! You need to specify it as a customid. So your user will look like this:

{
  customIDs: {
    stableID: stableId ?? ""
  }
}

Anonymous: Interesting, unfortunately this change made Next.js very angry and it crashed. I wonder if it’s because the stable id isn’t being set before this happens (though my assumption would be that the empty string might protect against a fatal error?).

tore (Statsig): Any exception trace?