Frequently Asked Questions

Currently, there isn't a feature to 'freeze' a gate in the way described. When a feature gate's status is changed to 'Launched' or 'Disabled', it will always return the default value (TRUE for Launched, FALSE for Disabled) for all users, regardless of whether they are new or old. The gate will stop generating billable exposure events, and you'll stop incurring costs. The gate reference is likely safe to be cleaned up in the codebase.

Implementing a feature to remember the previous evaluation for a gate would be challenging. However, if you want this sort of behavior, you could implement it yourself. If you have a client-side app, it would be fairly easy to store someone's evaluations in local storage, and check for a value there before checking Statsig.

For server-side implementation, you would need a database and a bit more work. This would allow you to store the evaluations and check for a value before checking Statsig, effectively creating a 'freeze' behavior.

Please note that these are workarounds and not built-in features of Statsig. If you have further questions or need assistance with a specific use case, please reach out for further assistance.

At present, there is no direct support for configuring the recipient of feature gate emails to an entire team instead of just the individual who created or edited it. However, there is an alternative way to receive notifications for a gate. Users can "subscribe" to them by clicking on the subscribe button. This will allow them to receive notifications related to the feature gate. Please note that this is the only other way to receive notifications for a gate currently supported.

When you click on an event in the Event Log Stream, a popup with detailed information, including the stableID, should appear. This is the default behavior when the ID type configured on the gate/experiment is shown in the list view.

If you want to make a gate or experiment a stableID based gate/experiment, the most relevant column to show to ensure you have configured your gate/experiment and checks in code correctly is the unit type of the gate/experiment itself. If you change the unit type to “stableID”, that column will change to stableID.

It's important to note that Statsig allows you to experiment on userID, stableID, and up to 5 customIDs. Therefore, the table can't really scale to show all the IDs inline. However, once the unit type is changed to "stableID", the Statsig Stable ID will appear in the exposure stream.

The tutorial on setting up an A/B/n test using Statsig's Experiments feature uses two examples to illustrate different variants you might test in an experiment.

The first example, enable feature, is a boolean type feature gate that can be either true or false. This represents the feature being on or off and is a simple A/B test scenario where you're testing the impact of a feature being enabled vs. not enabled.

The second example, Circle/Square, is a multi-variant scenario where you're testing more than two variants. In this case, you're testing the impact of showing a circle vs. a square shape in your product.

The screenshots in the tutorial are meant to illustrate the process of setting up these tests in the Statsig console. If you're finding the tutorial confusing, it is recommended to follow along in the Statsig console as you read. This should help clarify the process.

The tutorial is designed to use an A/A test as an example to show you how to set up any A/B test easily, of which A/A test is a special case. However, we acknowledge that it can seem a little confusing and we are committed to improving it for the future.

If you have any specific questions about the tutorial or any other aspect of using Statsig, feel free to ask. We're here to help.

Switching to the native Statsig event logger does not inherently change the synchronization time to Pacific Standard Time (PST), as the synchronization schedule is consistent regardless of the logging method. However, the Statsig client SDK does facilitate more real-time data capture by automatically flushing logged custom events to the Statsig servers every 10 seconds.

For experiment results, Statsig has implemented a more frequent refresh window. During the first day of an experiment, results are computed hourly. After the initial 24-hour period, the results are then refreshed on a daily basis.

It is important to note that these computations are based on a day defined from midnight to midnight PST. For businesses requiring synchronization in Coordinated Universal Time (UTC), Statsig offers the ability to switch to UTC, but this is an option available exclusively for Enterprise customers. This is due to the additional costs associated with maintaining a separate pipeline for UTC synchronization.

The request body for feature gates in the React client SDK is obfuscated and encoded in a specific way that is intended to be processed on the server side by Statsig. This is a security measure implemented to protect the integrity of the data. As such, it is not designed to be decoded on the client side and there is no supported method for doing so.

If you need to inspect the values of your feature gates, it is recommended to do so through the Statsig console. This is the most reliable and secure method to access and manage your feature gates. Please refrain from attempting to decode the obfuscated content on the client side as it may compromise the security and integrity of your data.

When a vulnerability alert is triggered in Snyk for the Statsig Ruby SDK due to the MPL-2.0 license, it is important to understand the implications of the license and how it may affect your organization. The Mozilla Public License 2.0 (MPL-2.0) is a widely accepted open-source license that allows the code to be used in both open-source and proprietary projects.

Under MPL-2.0, if modifications are made to the original code, those changes must be disclosed when the software is distributed. However, the MPL-2.0 license permits the combination of the licensed code with proprietary code, which means that the SDK can be used in closed-source applications without requiring the entire application to be open-sourced.

It is important to note that while the MPL-2.0 license is generally compatible with other licenses, it may be flagged by security tools like Snyk because it requires review and understanding of its terms. Organizations should consult with their legal team or open-source compliance experts to ensure that the use of MPL-2.0 licensed software aligns with their policies and legal obligations.

If your organization has made no modifications to the original code, there should be no concerns regarding the need to disclose changes. Ultimately, the decision to use software under the MPL-2.0 license should be made by the decision-makers within the organization after careful consideration of the license terms and compliance requirements.

In Statsig, you can manually change the default value of a parameter in a layer. This can be done from the layer's homepage. Here's how:

1. Navigate to the layer's homepage.

2. Locate the parameter whose default value you want to change.

3. Update the default value as needed.

Please note that parameters in a layer are designed to exist across multiple experiments in the layer. Therefore, the layer's homepage is the place where they live.

Remember, the ability to change the default value of a parameter in a layer allows you to manage your Statsig integration code more efficiently. It's one of the benefits of using default parameters in Statsig.

If you're looking for a way to change the default value at the level of an experiment in the layer, where the parameter is first defined, please note that this is not currently possible. The parameters are defined at the layer level, not at the experiment level.

We hope this information helps you in managing your parameters in Statsig. If you have any more questions, feel free to reach out.

When an experiment decision has been made and there is a need to continue collecting metrics without resetting user assignments, the platform does not offer a native solution to directly address this scenario.

Once a decision is made on an experiment, the platform does not support undoing that decision. The 'Reset' option is available; however, it will revert the experiment to an unstarted state, and all users will receive the default experience. If the experiment is reset or a new experiment is started, users will be randomly assigned to a group, which may differ from their original assignment.

It is important to note that resetting an experiment will not preserve the initial user group assignments, which may be critical for long-term metric collection and analysis. To monitor the performance of long-term metrics after an experiment has ended, the Metrics Explorer can be utilized. While it does not provide confidence intervals and p-values, it does offer absolute numbers for each metric across all groups in the experiment.

For future experiments, it is recommended to enable Reviews in the project settings to provide better oversight and prevent similar situations. Reviews offer guardrails that can help in managing changes that impact production environments.

To edit the returned JSON for a dynamic config after adding a targeting rule, locate and click on the 'Edit' link adjacent to the 'Returns JSON' text. This action will bring up the JSON configuration editor.

Within this editor, you can freely modify the configuration parameters and their corresponding values as required by your application. After making the necessary changes, ensure to save your updates by clicking on the 'Confirm' button.

This process allows you to tailor the dynamic config to suit the specific needs of your application, even after targeting rules have been applied. If you encounter any difficulties or the editor does not load as expected, please reach out for further assistance.

Enabling a metrics ingestion pipeline in Statsig involves a series of steps to ensure that your data is correctly integrated and utilized for analysis.

Begin by connecting your data warehouse to Statsig, which will require read access to metric and exposure data, write access for results and exposures, and the ability to run jobs and query data. Once connected, define your metric and assignment sources according to Statsig's expected schemas to map the data accurately.

Metrics can then be ingested through various mechanisms, including Server and Client SDKs, HTTP API, Integrations, Data Warehouse Ingestion, or the Data Warehouse Native Solution for enterprise contracts.

If you encounter any issues during the ingestion process, utilize the Metrics Dashboard in Statsig for debugging. After successful data import, you can proceed with feature gate or experiment implementation.

For detailed guidance on connecting your warehouse and defining sources, refer to the Statsig documentation.

Ensuring consistent experiment results across Client-Side Rendered (CSR) and Server-Side Rendered (SSR) pages when using Statsig involves establishing a single source of truth for user assignments. The challenge arises when independent network requests on different pages result in varying initialization times, potentially leading to different experiment variants being presented to the user.

To address this, it is recommended to have one consistent source of truth for the user's assignment. This can be achieved by using the assignments from the first page (CSR) and carrying them over to the second page (SSR), rather than initializing a new request on the second page.

This approach ensures that the user experiences a consistent variant throughout their session, regardless of the page they navigate to. It is important to note that the user's decision for any getExperiment call is determined at the time of initialization, not when getExperiment is called. The getExperiment call simply returns the value fetched during initialization and logs the user's participation in the experiment and their assigned group.

To implement this, developers should ensure that the assignments are fetched from the network request on the first page and then passed to the second page, avoiding the need for another initialization request and the management of an ever-growing list of parameters.

To retrieve currently active experiments using JavaScript in the browser console, you can use the Statsig.getExperiment or Statsig.getLayer methods. However, these methods require you to know the name of the experiment or layer. There is no documented method to retrieve all active experiments without knowing their names.

For group overrides, there is no documented method to retrieve group IDs directly. However, you can use the Statsig.overrideConfig method to override the value of a specific experiment for a specific user. The value you provide should match the structure of the experiment's parameters. Here is an example:

javascript Statsig.overrideConfig("a_config_or_experiment_name", { key: "value" }, "a_user_id"); In this case, { key: "value" } should match the structure of the parameters for the experiment you're trying to override. There is no documented way to override groups directly using group IDs.

If you're unsure about the structure of the experiment's parameters or need further assistance, it is recommended to reach out to the Statsig team for more specific guidance.

Please note that it may not be possible to override the group. You can only override the config object that's returned by a getExperiment call.

Integrating the Statsig SDK server-side in a Next.js application without a custom server or Vercel deployment can be achieved by utilizing the register API to initialize the SDK on server startup.

This method allows for the SDK to be set up before serving requests, ensuring that feature flags and configurations are available for server-side rendering. However, it is important to note that the register API may only trigger on the first request received by the server.

To ensure the SDK is initialized, a health check or similar request may be necessary to activate the register process.

Additionally, depending on the hosting solution for the Next.js app, writing a data adapter to read Statsig configurations from a cache could be a viable alternative. This approach would involve creating a custom adapter that interfaces with a caching mechanism to retrieve the necessary configurations for Statsig.

For more detailed guidance on implementing feature flags with Next.js, refer to the official Statsig documentation.

Yes, it is possible to preserve the Statsig Stable ID across different platforms such as mWeb and mobileApp. The Stable ID is generated for anonymous use cases and is persisted in local storage. For signed-in users, consistency is ensured across different surfaces, such as mobile app vs desktop, as long as there's a userID used across them.

If you're experiencing issues with StableID changing across subdomains or platforms, one possible solution is to store the StableID in a cookie and reuse it. You can store and retrieve your own id and use the overrideStableID method on the SDK to send it to Statsig. Your overridden id will always take preference. This way, you can set the StableID for not logged in users and your in-house id for people that are logged in.

Here's an example of how you can override the StableID:

javascript await Statsig.initialize(  "client-sdk-key",  null,  {    overrideStableID: '<passed via cookie or via db lookup>'  } );

This should ensure that both your marketing site and main app use the same StableID, allowing you to run experiments that span both sites.

If you're experiencing a continuous network error while running Statsig 0.23.0 in a Python 3.9 environment, it's important to first understand the environment in which you're running the Statsig SDK and the version you're using.

If the error is occurring locally and continuously, the Statsig SDK will keep retrying and returning the same error. To debug this, you can send a curl command to our events API from your local environment. Here's an example of how to do this:

bash curl  -X POST \  'https://events.statsigapi.net/v1/log_event' \  --header 'statsig-api-key: <YOUR SDK KEY>' \  --header 'Content-Type: application/json' \  --data-raw '{"events": [{"user": { "userID": "1" }, "time": 1667512336, "eventName": "test_event"}]}'

This command sends a test event to the Statsig events API. If this command works, it means your local environment can successfully send requests to the Statsig API. If it doesn't, you may need to check your local network settings, such as DNS or ad-blockers, which could be blocking requests to the API.

Remember, the Statsig SDK is designed to retry sending events if errors are encountered. If you're still experiencing issues after trying the above steps, please provide more details about your environment and the error you're encountering for further assistance.

To test whether a metric based on a custom event is tracked in non-production environments, you can follow a specific process. While non-production events are visible in diagnostics, they are not included in Pulse results.

To include these events, you can log them as regular events. It is important to note that non-production data is intentionally not included in metrics to prevent non-production test data from being included in the metrics. When testing your event logging, you can suffix your events with '_test' and use the production environment. Once you are ready to finalize your event logging, you can remove these suffixes.

If you encounter issues where the Metrics Logstream is not visible, it may indicate a connection or schema issue, and Statsig is not receiving your precomputed metrics. In such cases, you should reach out for assistance, and a team member will provide support.

If you encounter an error message stating This cohort has failed to sync because the API key that was provided no longer has access to the partner. Please check the access level on the partner's website concerning this integration. while integrating Amplitude with Statsig, it suggests that there might be an issue with the Server Secret Key you're using.

Here are a few steps you can take to troubleshoot this:

1. Double-check the Server Secret Key: Ensure that you've copied the correct Server Secret Key from the API keys page in your Statsig Project Settings.

2. Check the key's access level: Make sure that the Server Secret Key you're using has the appropriate access level for the integration.

3. Re-enter the key: Try re-entering the Server Secret Key in Amplitude and save the changes.

If you've checked these and the issue persists, it might be a more complex issue. In this case, it is recommended to reach out to the Statsig support team for further assistance.

When considering the removal of preexperimental bias from a feature that was rolled out with limited initial data, resalting the feature gate is not specifically recommended for this purpose.

Pre-experimental bias can occur by chance and may not always significantly impact the results of an experiment. If the overall difference in metrics due to this bias is small, it might not meaningfully affect the interpretation of the results.

In such instances, the bias can be acknowledged as a cautionary note, and the affected metrics should be treated with a degree of skepticism. To address concerns about pre-experimental bias, it is advisable to utilize the 'Days Since Exposure' view.

This tool assists in identifying any novelty effects and pre-existing experiment effects. Should bias be detected, users will receive a notification, and a warning will be displayed on the relevant Pulse results. This approach allows for a more informed analysis of the feature's impact without the need to resalt the feature gate.

In Statsig, the "Days since exposure" graph in the View Details section can be utilized to understand if an experiment is pulling forward an event. This graph displays the cumulative lift of a metric over time since a user's first exposure to an experiment. If you observe a significant lift immediately after exposure that diminishes over time, this could suggest that the experiment is pulling forward events.

However, it's important to note that there isn't a built-in tool specifically designed to identify pulled forward events. The closest feature available is a toggle in the top right of the scorecard that allows you to select “Days Since Exposure” instead of “Cumulative”. This feature enables you to look across all metrics inline.

Remember, the interpretation of the "Days since exposure" graph and the use of the toggle feature require a good understanding of your metrics and the potential impact of your experiments.

The statsig::app_error event is an automatic log created by the Statsig SDK when it encounters an uncaught exception on your webpage. This event is not necessarily related to your Statsig implementation, but rather, it is associated with any error on your webpage.

The statsig::app_error event can be particularly useful when you navigate to the Event Logstream and inspect the details. Depending on the nature of the error, the SDK may be able to capture the line and error message.

It's important to note that the occurrence of this event on iOS devices could be due to a specific condition or issue unique to the iOS environment. However, without more specific information, it's difficult to provide a definitive answer.

For further assistance with this issue, it is recommended to reach out to the Statsig team directly. They may be able to provide more insight into this specific event and why it's being triggered on iOS devices.

When deploying to an environment in Statsig, if you encounter an issue where the RULE is always Default and the REASON is Unrecognized, it typically means that the SDK was initialized, but the config or feature gate you're trying to evaluate did not exist in the set of values. This could be due to a few reasons:

1. The feature gate or dynamic config you're trying to evaluate does not exist or is not correctly spelled in your code. Please double-check the spelling and case-sensitivity.

2. The SDK might not have been able to fetch the latest rules from the Statsig server. This could be due to network issues or if the SDK initialization did not complete successfully.

3. If you're using a server SDK, it's possible that the SDK is outdated and doesn't recognize new types of gate conditions. In this case, upgrading the SDK might resolve the issue.

Remember, the Unrecognized reason is only given when the SDK is initialized, but the config or feature gate did not exist in the set of values.

It's also important to ensure that you are waiting for initialize to finish before making evaluations. For instance, calling checkGate inside the callback or after you are sure the callback has been triggered.

Additionally, check if the environment you've deployed to is able to make requests to the relevant URLs. In some cases, these requests might be blocked by the client in your production environment.

If you're still having trouble, please provide more details about your setup and the issue, and a Statsig team member will assist you shortly.

If you are using the synchronous provider and Server Side Rendering (SSR), the assignment reasons chart should be 100% with the reason “Bootstrap”. There should not be any “network”/“Cache”/etc. If you are seeing a lot of uninitialized values, it could point to a potential implementation issue.

One possible cause could be changes in the user object that you pass into the synchronous provider. If there are fields on the user object that you are loading asynchronously or in an effect elsewhere, it could trigger the SDK to update the values for the user.

To debug this issue, you should verify each of the render passes on the StatsigSynchronousProvider and ensure that there is only a single render with a static user object. If the user object changes and it rerenders, it could lead to the issue you are experiencing.

If you are calling useExperiment() in your code, make sure that the user object passed into the provider is static and never changes. If the user object changes after you bootstrap the StatsigSynchronousProvider, it will cause the provider to re-fetch initialize values using a network request, effectively discarding the results passed to it through SSR.

It's also important to note that if you are using SSR correctly, you should never have a “network” reason. If you are seeing a “network” reason, it could indicate that your users are only going through the Client Side Rendering (CSR) flow.

If you're using the dynamicConfig.get("key", defaultValue) method to fetch a value from a Dynamic Config in Statsig and receiving null, it could be due to several reasons:

1. The key does not exist in the Dynamic Config: Ensure that the key you're trying to fetch is correct and exists in the Dynamic Config.

2. The Dynamic Config has not been initialized or fetched correctly: Check the initialization and fetching of the Dynamic Config. Make sure it has been done correctly as per the Statsig documentation.

3. Issue with the default value provided: There might be a problem with the default value you've provided. Verify that it is correct and compatible with the expected value type.

If you've verified the key, checked the initialization and fetching of the Dynamic Config, and ensured the default value is correct, but you're still receiving null, it might be a more complex problem that requires further investigation. In such cases, we recommend flagging this issue for further assistance from the Statsig team.

The "Uninitialized" assignments in Statsig could be due to the initTimeoutMs parameter in the Provider. This parameter determines how long the Statsig client waits for the initial network request to respond before proceeding. If this timeout is too short, it could lead to an "Uninitialized" status if the initialization network request doesn't complete within the specified time.

In some cases, a 400ms timeout might be too short, causing the initialization to not complete in time, especially if network conditions are not optimal. It is recommended to consider increasing this timeout value to allow more time for initialization.

However, for Server Side Rendering (SSR) pages using the StatsigSynchronousProvider, there is no concept of initTimeoutMs. Therefore, the initialization happens on the server and there should be minimal uninitialized cases. If you are seeing a notable number of uninitialized cases for an experiment on an SSR page, it could be due to other factors not related to the initTimeoutMs parameter.

Please note that the initialization process involves a network request, and depending on the network condition and location of the users, a short timeout like 400ms might not be sufficient for many users.

In order to enable reviews for changes that will impact your production environment, you need to follow certain steps. This includes changes to a dynamic config or gate, which affect rules with the production environment or apply to all environments. It also includes changes to an experiment that has started.

The philosophy behind this approach is to strike a balance between allowing developers to move quickly in a local environment, while ensuring that any changes that will impact end users will require a review.

Here's how it works:

1. Reviews are only required for changes that will have an effect on the production environment. This means that if you're making changes to a dynamic config or gate, reviews will only be required if these changes affect rules with the production environment or apply to all environments.

2. For experiments, reviews are only required for changes to an experiment that has started. This is because these changes could potentially impact your end users.

If you're seeing behavior that doesn't align with these rules, it's possible that there's a technical issue that needs to be addressed. However, in most cases, following these guidelines should ensure that your changes are reviewed appropriately before they're implemented in a production environment.

When conversions are not showing up in Statsig test results, there are several troubleshooting steps you can take to resolve the issue. First, verify that the events sent from your application include the same identifier (either userID or stableID) as the one used in your client SDKs for the experiment. This ensures that Statsig can correctly join the events with the appropriate experiment groups.

Additionally, be aware that Statsig computes custom metrics on a daily basis for your Metrics dashboard and aggregates them for the duration of the experiment in your Pulse Results. Custom metrics will not populate until the day after creation and will not retroactively apply to previous days.

Statsig will calculate them from the creation date forward. If you are using precomputed metrics, confirm that Statsig is receiving your precomputed metrics. If the Metrics Stream is not visible at the bottom of your Metrics Catalog, there may be a connection issue or an invalid schema.

If these steps have been checked and the issue persists, it is advisable to contact a Statsig team member for further investigation.

When a feature gate is set to Launched or Disabled, it will always return the default value (TRUE for Launched, FALSE for Disabled) and stop generating billable exposure events. However, disabling a feature gate will not stop checks against it - it'll just always return false. Removing it from your codebase will stop checks - assuming you don't have any old versions of your code still running anywhere.

If you're still seeing checks after that, it's possible that there might be some instances where the gate is still being referenced. Double-check your codebase to ensure that all references to this gate have been removed. If the issue persists, it is recommended to flag this issue to the Statsig team for further assistance.

In some cases, you might still see exposure events in Stitch for gates you've launched/disabled (and all are presumably False). This could be due to a bug in the system. The Statsig team is always ready to identify and fix such issues.

Please note that fixes for such bugs are usually rolled out promptly and should resolve over a short period of time. However, if you're still not seeing the issue as resolved on your end, it is advisable to reach out to the Statsig team for further investigation and assistance.

When comparing event counts between Statsig and GA4, it is important to consider the potential for discrepancies due to time zone differences.

Statsig operates on a default day that starts at GMT-8 (Pacific Standard Time) and maintains this standard throughout the year, without adjusting for daylight savings time. This can lead to differences in daily event counts when compared to other platforms that may use different time zones for their reporting.

Additionally, the methodologies for logging and processing events can vary between Statsig and GA4, which might contribute to the observed discrepancies. It is also crucial to check for any filters or exclusions that may be applied within either platform, as these can affect the event counts.

If discrepancies continue to be significant, a thorough review of the event logging code is recommended to ensure that events are being logged accurately and consistently across both platforms. In cases where the issue persists, reaching out to the Statsig support team for further assistance is advised.

When utilizing the Statsig PHP SDK to send gate exposure data, it is expected that the events should appear in the Diagnostics event stream for the corresponding gate once the cron job is executed.

To ensure that the SDK is properly initialized and the cron job is configured correctly, developers should review their implementation. A typical initialization snippet and cron job usage are essential for troubleshooting any issues with event visibility on the Statsig dashboard.

If events are not appearing as expected, developers should verify that the SDK is correctly set up and that the cron job is functioning as intended.

It is important to ensure that the SDK is properly flushed before the process exits to guarantee that all events are sent to Statsig servers. If discrepancies persist after verification, further investigation into the SDK's event logging mechanism may be necessary.

In the event that some metrics in your ongoing experiment are showing as "unavailable", it could be due to an issue with how your events are being logged or matched with the experiment groups. To resolve this, ensure that the events you're sending include the same identifier (either userID or stableID) as the one used in your client SDKs for the experiment. This will help Statsig join the events with the correct experiment groups.

It's important to note that exposures on the last day (the day you made a decision) are not counted as exposures. Therefore, when analyzing your data, remember to filter out that day.

If you're still experiencing issues, it might be helpful to check your query history for the user to understand which data is being pulled, and if queries are not executing or are failing.

In some cases, metrics may temporarily show as unavailable due to Statsig recomputing the metric for the latest numbers. However, these metrics should become available once the final aggregated data is uploaded and any issues with this process are resolved.

If the issue persists, please let us know and we'll have a Statsig team member assist you further.

If you have integrated Statsig with Segment and are experiencing issues with some experiment events not being sent to Segment, there are a few things you could check:

1. Event Filtering: Statsig allows you to filter which events should be sent to Segment. Please check your Event Filtering settings in the Statsig console to ensure that the missing events are not being filtered out.

2. Segment Integration: Please ensure that your Segment integration is still correctly configured and enabled.

3. Event Data: Ensure that the events you're sending from Segment include the same identifier (either userID or stableID) as the one used in your client SDKs for the experiment. This will help Statsig join the events with the correct experiment groups.

If you've checked all of these and are still experiencing issues, please let us know. We'll flag this issue and a Statsig team member will be by to help shortly.

When user interactions with a feature, such as clicks on a banner, are not reflected in the Statsig exposure export, it is important to understand that Statsig exports all exposures without exception. If a user has been exposed to a feature gate, they will be included in the analysis. However, discrepancies can occur for several reasons.

For instance, if the user's attributes have changed post-exposure or if the feature gate rules were modified, the user might not qualify anymore but would still be included in the analysis.

To verify exposures, users can log into the Statsig console, select the feature gate, and click on 'View Pulse'. The 'Cumulative Exposures' panel will display the total exposures, including a breakdown of users exposed to the feature (treatment or Pass group) and users not exposed to the feature (control or Fail group).

If issues persist, it is recommended to utilize the debugging tools provided in the Statsig documentation to understand why a certain user received a specific value. In cases where the problem cannot be resolved, reaching out to the Statsig team for assistance is advised.

In response to concerns about an unexpected increase in billable events, it is important to note that Statsig has robust mechanisms in place to prevent the duplication of billable events.

For client SDKs, Statsig ensures that only one billable unit is counted within a one-hour window, while for server SDKs, this window is reduced to one minute. This effectively means that multiple exposure events for the same stable ID within these time frames do not lead to additional billable units. An increase in billable events could potentially be attributed to a rise in unique user interactions or an uptick in the usage of feature gates, experiments, or dynamic configs.

Users are advised to monitor their billing through the billing page under project settings, which provides aggregated information at the gate/test level. In cases where discrepancies are observed, reaching out to the Statsig team is recommended for a detailed investigation.

Additionally, it has been noted that for certain plans with unlimited gate checks, the plan may not have been correctly applied to a project. Once rectified, the billing page will reflect a significant number of events left, with most gate checks being free.

It is also clarified that only gate checks on rules that are partially open and have analytics enabled are billed.

If you're unable to adjust the dates in the scorecard for your old experiment, it could be due to the experiment's end date being set in the past. When an experiment is created, it is marked with a specific completion date. If you want to extend the experiment or adjust the dates, you have two options:

1. Clone the existing experiment and restart it, making sure to set the experiment end date to the desired date or mark it as ongoing.

2. Reset the experiment from the same menu and restart it from there.

After resetting the experiment, you can change the end date. However, you might notice that the pulse loading takes a longer time than usual. This is not normal even for this experiment. The Wall time, which is the total "real time" it took to run the job, from when it started to when it finished, might be longer than the job time, which is the actual compute time run on your warehouse. This could be due to transient network errors communicating with your warehouse, which is unusual. If you encounter this issue, it's recommended to flag it for assistance.

In the Explore tab of Pulse Results, you can use user dimensions to drill down and explore results. However, the user properties you can segment on are only derived from exposure events, not custom events. Any overrides that are used to force a user into the test will be ignored from pulse results. If you only have override userIds in the results, nothing will show up in the pulse filters. If you're having trouble finding a specific user property, it might be due to these reasons.

It's also important to note that the availability of User Dimensions can vary depending on the gate/experiment you’re querying on. Some entities have User Dimensions available, some don’t based on whether there are exposures or not.

Regarding the issue of error ranges for Daily Topline Impact and Daily Projected Launch Impact not populating, this was due to a bug that occurred during our internal migration of infrastructure and data. We have fixed this bug and it affected certain metric types. All new custom queries should not have a problem. For some metrics, the topline impact could have a 0 as the CI, because all users can have the value of 1 if the metric is an indicator. However, we'll do an audit to make sure the calculation is consistent with our scorecard.

When attempting to update user attributes in a development environment using the JavaScript SDK and subsequently searching for the user in the Statsig admin UI, it is important to understand the expected behavior and requirements for user data to be logged and visible.

Users initialized as anonymous will only have a stableID. If you assign them to a feature or test, they will appear in the Users tab, searchable by stableID, although there may be a delay as data takes time to reach the Users tab.

To see a user's profile updated with additional information such as userID and email in the Users tab, you must enrich their user object with these details and then trigger a subsequent event. It is important to note that only 'production' data is displayed in the Users tab.

However, you can view assignments in real-time for both production and lower environments in the Diagnostics tab. To log the user object, you must call the logEvent API or alternatively use checkGate or getExperiment, as merely initializing the SDK will not log the user object.

For debugging purposes, you can use the Exposures Log stream for any test or gate under the Diagnostics tab, and the Event logstream under the Metrics page to view any custom logEvent events sent to Statsig.

In the scenario where config.getGroupName() returns null in your SSR page using NextJS and Statsig, it's important to understand that the getGroupName() function is expected to return null if the dynamic config does not have a group name associated with it. This could occur if the dynamic config is not part of an experiment group.

If you believe this is an error and the dynamic config should be part of a group, it's recommended to double-check your configuration in the Statsig console. If the issue persists, it might be a good idea to flag this for further investigation by the Statsig team.

In the case where you're calling getGroupName on an experiment config, such as statsig.getExperiment('test').getGroupName(), and it returns null or "undefined", it's possible that the groupName is not included in the initializeValues payload.

This issue might require a fix to the NodeSDK. The NodeSDK downloads the groupname via Statsig on initialization, but it doesn't include it in the payload generated by Statsig.getClientInitializeResponse.

As a workaround for now, you can create a group name parameter in your experiment or config, and get that from the payload of the dynamic config. This should help you bypass the issue until a fix is implemented.

In Statsig, the "Users" tab is designed to display common properties and evaluations for a given user ID. However, it may not always show all custom properties. The most reliable way to view custom fields added to the user from the Statsig dashboard is through exposure checks.

If you are able to group by a specific user property when performing a custom query for a gate, it indicates that the property is being logged correctly. The absence of this property in the "Users" tab could be due to the temporary nature of custom properties.

If you encounter any issues or require further clarification, please reach out for assistance.

In Feature Gate and Dynamic Config rules, the "equals" operator is not explicitly supported for numbers. However, there are alternative methods to achieve an equivalent functionality.

One such method is to use the "any of" operator with a single item in the array. This will effectively perform an equality comparison. Importantly, there is no need to cast integers to strings for this operation; the system handles the necessary conversions.

As a best practice, it is recommended to test this functionality to ensure it meets your specific requirements.

Remember, the operators provided in Feature Gate and Dynamic Config rules are designed to offer flexibility and cover a wide range of use cases. For numbers, operators like "greater than", "less than", "greater than or equal to", and "less than or equal to" are often more useful in defining conditions. If you need to check for an exact match, you could potentially use both "greater than or equal to" and "less than or equal to" with the same value to mimic an "equals" operation.

For any further questions or specific functionality needs, please reach out to the support team.

To check if a specific user ID or stable ID has been manually added to any experiments using "Manage Overrides", follow these steps:

1. Log into the Statsig console

2. On the left-hand navigation panel, select "Users".

3. In the User's tab, enter the specific user ID or stable ID you're looking for in the "ID" field and select the appropriate "ID Type" from the dropdown menu.

4. Click "Submit" to load the data for the entered ID.

The resulting information will include a section titled "Overrides", which lists all the entities (Feature Gates, Experiments, etc.) that the given ID has been manually overridden into. Please note that you need to enter both the ID and ID type to submit a query.

If a stable ID is not showing in search results despite being present in many experiments, there are a couple of things to check:

1. Ensure that the box “include in Pulse results” was checked in the override modal.

2. Consider the timing of the assignments/exposures. It may take some time for the system to update and reflect recent changes.

If you're only seeing overridden values in the Exposure Stream for your experiment, there could be several reasons for this. Here are some steps you can take to troubleshoot:

1. Check the Initialization Status: Each hook has an isLoading check, which you can use, or the StatsigProvider provides the StatsigContext which has initialization status as a field as well. This can be used to prevent a gate check unless you know the account has already been passed in and reinitialized.

2. Check the Data Flow: Ensure that the id_type is set correctly and that your ids match the format of ids logged from SDKs. You can check this on the Metrics page of your project.

3. Check the Query History: If your data is still not showing up in the console, check your query history for the user to understand which data is being pulled, and if queries are not executing or are failing.

4. Check the Exposure Counts: If you're seeing lower than expected exposure counts, it could be due to initializing with a StatsigUser object that does not have the userID set, and then providing that on a subsequent render. This causes the SDK to refetch values, but logs an exposure for the “empty” userID first. To prevent this, ensure the userID is set before initializing the StatsigUser object.

If you've checked all these and the issue persists, it might be best to reach out for further assistance.In some cases, users may still qualify for the overrides based on the attributes you’re sending on the user object. This may be due to caching, or it may be due to the user qualifying for other segments that control overrides.

For example, if users have "first_utm_campaign": "34-kaiser-db" being sent on the user object, they would qualify for a segment that’s being used in the overrides.

It's also important to note that overridden users will see the assigned variant but will be excluded from experiment results. We have a way to include users in results for experiments not in layers, but it seems we don’t have that option for experiments in layers.

Lastly, consider why you are using overrides in this scenario instead of a targeting gate. Overrides can be used to test the Test variant in a staging environment before starting the experiment on prod. However, if some of your customers have opted out of being experimented on, a targeting gate might be a more suitable option.

In Statsig, the 'Last daily Event Count' column displays the count of events for each day. If you observe a value of 0 for all events, it indicates that no events were logged on the previous day. The numbers will update when new data is processed in a few hours. You can click on each event to see this.

However, it's important to note that the charts only include production events. Non-production events can be viewed in the logstream by toggling to non-prod logs. These non-production events are not included in the charts to prevent them from polluting experiment results.

If you're running an Autotune experiment and notice that the Success Event is not being counted on the Pulse Results, it could be due to a few reasons.

Firstly, ensure that the events you're sending include the same identifier (either userID or stableID) as the one used in your client SDKs for the experiment. This is crucial for Statsig to join the events with the correct experiment groups.

If you're still not seeing the expected results, you might want to check your query history for the user to understand which data is being pulled, and if queries are not executing or are failing. If data is loading, it's likely we're just processing. For new metrics, give it a day to catch. If data isn't loaded after a day or two, please check in with us. The most common reason for metrics catalog failures is due to id_type mismatches.

In the case of using a Segment event for the success metric, it's important to note that there may be a delay in the Autotune results being updated. Data should be updated hourly but could be a bit later depending on your attribution window.

If you continue to not see the data as expected after more time has passed, please reach out for further assistance.

The discrepancy between the experiment allocation counts and the ssr_search_results_page_view (dau) counts could be due to several reasons:

1. **User Activity**: Not all users who are allocated to an experiment will trigger the ssr_search_results_page_view event. Some users might not reach the page that triggers this event, leading to a lower count for the event compared to the allocation.

2. **Event Logging**: There might be issues with the event logging. Ensure that the statsig.logEvent() function is being called correctly and that there are no errors preventing the event from being logged.

3. **Timing of Allocation and Event Logging**: If the event is logged before the user is allocated to the experiment, the event might not be associated with the experiment. Ensure that the allocation happens before the event logging.

4. **Multiple Page Views**: If a user visits the page multiple times in a day, they will be counted once in the allocation but multiple times in the ssr_search_results_page_view event.

If you've checked these potential issues and the discrepancy still exists, it might be a good idea to reach out to the Statsig team for further assistance.

Another possible reason for the discrepancy could be latency. If there is a significant delay between the experiment allocation and the event logging, users might abandon the page before the event is logged. This could lead to a lower count for the ssr_search_results_page_view event compared to the allocation.

When encountering an unexpected spike in the new_dau metric without a corresponding increase in website traffic, there are several potential causes to consider. Firstly, it is important to verify the accuracy and consistency of event logging. Any discrepancies in how events are logged can lead to anomalies in the metrics.

Secondly, if data is being imported from external sources, ensure that the ingestion process is functioning correctly and that data is imported in a timely manner.

Additionally, be aware of timezone differences that may affect the interpretation of metrics; Statsig uses Pacific Standard Time (PST) as its default, which could lead to discrepancies if other systems are set to different timezones.

Lastly, the use of user identifiers should be consistent across all events and sessions to maintain the integrity of the data.

If these checks do not resolve the issue, further investigation into specific events that correlate with the metric increase may be necessary. It is also advisable to reach out directly for support if the cause of the spike remains unclear.

When conducting experiments, it is crucial to ensure that there is no cross-contamination between control and treatment groups and that data is accurately reflected in both funnel and summary views.

Cross-contamination can occur due to implementation issues, such as a race condition with tracking. This happens when users, particularly those with slower network connections, land on a control page and a page-view event is tracked before the redirect occurs.

To mitigate this, it is recommended to adjust the placement of tracking scripts. The Statsig redirect script should be positioned high in the head of the page, ensuring that it executes as early as possible. Meanwhile, page tracking calls should be made later in the page load lifecycle to reduce the likelihood of premature event tracking. This adjustment is expected to decrease discrepancies in tracking and improve the accuracy of experiment results.

Additionally, it is important to confirm that there are no other entry points to the control URL that could inadvertently affect the experiment's integrity. Ensuring that the experiment originates from the correct page and that redirects are functioning as intended is essential for maintaining the validity of the test.

Lastly, it is necessary to have specific calls in the code to track page views accurately. These measures will help ensure that the experiment data is reliable and that the funnel and summary views are consistent.

In order to get the useExperiment hook to work correctly in your code, you need to ensure that the StatsigProvider wraps your entire app or at least the root component that uses Statsig. This is because the StatsigProvider initializes the SDK and passes down its state to the rest of the React application via a StatsigContext.Here's how you can modify your code:

jsx // Initialize Statsig at a higher level, ideally in your app's root component <StatsigProvider  sdkKey=CLIENT_SDK_KEY  waitForInitialization={true}  user={{userID: userData?.id, email: userData?.email}}  options={{    environment: { tier: "development" },  }} >  <App /> {/* Your app's root component */} </StatsigProvider> // Then in your ButtonModal component export function ButtonModal({ ...props }: ButtonModalProps) {  const classes = useStyles();  const [ isButtonModalOpen, setIsButtonModalOpen ] = useState(false);  const { config } = useExperiment("button_test");  // Rest of your code... } By doing this, the StatsigProvider will initialize the SDK once and provide the necessary context for all child components. This should allow the useExperiment hook to work correctly. Remember, you cannot call useExperiment in the same component that is rendering the provider. The provider needs to wrap a component above this in the hierarchy.

In the context of experiments, when a user is attributed to the 'uninitialized' state, it indicates that the user has not been properly initialized within the experiment framework. As a result, the user will not be assigned to a real experimental group but will instead fall into the 'Default' group. It is important to note that the 'Default' group is not included in the analysis of experiment metrics.

Therefore, data from users in the 'uninitialized' state will not be included in the experiment metrics. This ensures the integrity of the experiment results by only including data from users who have been correctly initialized and assigned to a valid experimental group.

When integrating Statsig with Vercel's Edge Config, it is important to understand the behavior of user allocation during updates to experiment configurations.

According to our expert, increasing the allocation percentage in an experiment will not affect users who have already been assigned to the test. However, it is the decrease in allocation that could potentially remove users from the test upon their revisit.

For users who were not allocated during their first visit at a certain percentage, they can indeed be allocated if they revisit after the exposure allocation has increased. This is applicable when using a user ID as the identifier without considering sessions. It is also noted that the Statsig Client SDKs offer a keepDeviceValue argument with getExperiment that enforces stickiness via localStorage, although this does not apply to Server SDKs.

During an update delay, there might be a temporary mismatch in user allocation results when polling from the edge versus polling from the client SDK. The edge configuration might not reflect the new allocation immediately, leading to a period where the user could be allocated on the client side but not yet on the edge.

This mismatch will persist until the edge data is updated to match the new allocation configuration.