Skip to content

Update introduction.md for holdouts #3492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update introduction.md for holdouts #3492

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/holdouts/introduction.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ In the example below, the 1% Holdout is comparing the metric values of the 1% Ho
<img width="1254" alt="Screen Shot 2025-03-15 at 3 49 19 PM" src="https://github.com/user-attachments/assets/b6c3724d-784d-45e3-ad6e-230bd9d82214" />


## How to stop Holdouts
To stop holdout and let the users who were held-out see new features, you can choose to disable or delete the holdout. This will also stop tracking the effects of held-out features.

You can disable the holdout if you want to retain the results for further analysis or re-enable the holdout to apply to all selected configs again. Otherwise, you can delete the holdout if you are finished with analyzing the results and there's no need to re-enable the same holdout again.


## Best Practices
1. **Size** - Statsig recommends a small holdout percentage, say 1% – 2%, to limit the number of customers who don’t see new features.
2. **Duration** - Statsig recommends operating holdouts for a period of three to six months, and then releasing the holdout. Prolonging the holdout period may increase the complexity of your software as you’d have to maintain a functioning product with no new features for a longer period.
Expand Down