Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
jurajmajerik committed Jul 16, 2024
1 parent c986ed4 commit 04d495d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion contents/docs/experiments/experiment-significance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ To calculate significance, we measure p-values using a [Poisson means test](http

For your results and conclusions to be valid, any experiment must have significant exposure. For instance, if you test a product change and only one user sees the change, you can't extrapolate from that single user that the change will be beneficial or detrimental. This principle holds true for any simple randomized controlled experiment, such as those used in testing new drugs or vaccines.

Even with a large sample size (e.g., approximately 10,000 participants), results can still be ambiguous. For example, if the difference in conversion rates between variants is less than 1%, it becomes difficult to determine if one variant is truly better than the other. To achieve statistical significance, there must be a sufficient difference between the conversion rates given the exposure size.
Even with a large sample size (e.g. ~10,000 participants), results can still be ambiguous. For example, if the difference in conversion rates between variants is less than 1%, it becomes difficult to determine if one variant is truly better than the other. To achieve statistical significance, there must be a sufficient difference between the conversion rates given the exposure size.

PostHog computes this statistical significance for you automatically. We display on the results page when your experiment has reached statistically significant results, making it safe to draw conclusions and terminate the experiment.

Expand Down

0 comments on commit 04d495d

Please sign in to comment.