StatsTest Blog
Experimental design, data analysis, and statistical tooling for modern teams. No hype, just the math.
Reporting Templates: Stakeholder Language Without Overclaiming
Ready-to-use templates for presenting statistical results to non-technical stakeholders. Learn to communicate effect sizes, uncertainty, and practical significance without oversimplifying or overclaiming.
Robust Standard Errors: When to Use Them (and When to Use by Default)
A practical guide to heteroscedasticity-robust and cluster-robust standard errors. Learn when standard errors are wrong, which corrections to apply, and whether to use robust standard errors by default.
Robust Statistics Toolbox: Trimmed Means, Winsorization, and Rank Methods
A practical guide to robust statistical methods that work without normality assumptions. Learn when to use trimmed means, Winsorization, M-estimators, and rank-based tests.
Sample Ratio Mismatch: Detection, Root Causes, and Solutions
How to detect sample ratio mismatch (SRM) in A/B tests, understand its common causes, and what to do when your experiment groups have unexpected sizes.
Sequential Testing: How to Peek at P-Values Without Inflating False Positives
Learn how sequential testing methods let you monitor A/B test results as data accumulates while maintaining valid statistical guarantees. Covers group sequential designs, always-valid inference, and practical implementation.
Statistically Significant but Meaningless: Practical Thresholds for Evals
A 0.5% accuracy improvement with p<0.001 is real but worthless. Learn how to distinguish statistically significant from practically meaningful in model evaluation.
Time-to-Event and Retention Analysis: Survival Methods for Tech
A comprehensive guide to survival analysis for product analysts. Learn Kaplan-Meier curves for retention, log-rank tests for comparing groups, Cox regression for understanding drivers, and how to handle the unique challenges of tech product data.
Time-to-Event Sample Size: Practical Approximations
A practical guide to sample size calculations for survival studies. Learn how to power time-to-event analyses, what drives the sample size, and practical approximations for retention experiments.
Data Transformations: When Log, Sqrt, and Box-Cox Help vs. Mislead
A practical guide to data transformations in statistical analysis. Learn when transformations fix problems, when they create new ones, and how to interpret results correctly.
Testing Trends Across Ordered Groups: Jonckheere-Terpstra and Alternatives
When your groups have a natural order (dose levels, experience tiers, usage intensity), standard ANOVA ignores this structure. Learn about trend tests that leverage ordering for more power.
Two-Way ANOVA vs. Regression: Understanding Interactions for Product Teams
When to use two-way ANOVA versus regression for analyzing experiments with multiple factors. Covers interactions, main effects, and practical interpretation for product analytics.