Stratified sampling is a method of sampling that involves dividing the entire population into homogeneous groups known as strata (plural for stratum). Random samples are then selected from each stratum.

For example, if you had customers of different sizes (XS and XL) and randomized them into two groups - Control and Test, you'd want both Control and Test to be balanced across XS and XL customers.

Read also: Introducing Stratified Sampling

Stratified sampling is particularly useful in B2B scenarios and other relatively low volume or high variance scenarios to ensure this balance.

Automated Stratified Sampling

Automated stratified sampling is a feature that is currently in development. It aims to automate the process of stratified sampling.

Manual Assignment for Stratified Sampling

In manual assignment for stratified sampling, when setting up an experiment, you can configure overrides (e.g., force user X or Segment A into Control, force user Y or Segment B into Test). This is meant for testing; overridden users are excluded from experimental analysis in Pulse results.

If you do want manual assignment for stratified sampling, you should check the Include Overrides in Pulse checkbox. This will include the users you've manually overridden into each variant in all metric lift analyses. You can configure 100% of experiment participants into your test variants manually, or configure some subset of participants into variants manually and randomly assign the rest of your participants.

When you use the Statsig SDK for assignment, it takes care of randomization. When you control assignment of users, you're responsible for making sure users are balanced across experiment groups.

For more information, you can refer to the Stratified Sampling | Statsig Docs.