The bias surfaced when I segmented usage by cohort and compared adoption curves across regions and tenure. Engagement dropped sharply outside the early-access group. To validate impact, I reran the analysis with stratified sampling and reweighted the data to reflect the actual user distribution. The conclusions changed materially some features were far less “sticky” than initially believed. We adjusted by separating roadmap decisions for core vs. advanced users and added guardrails in future analyses to always sanity-check demographic and behavioral balance before trusting topline metrics.