Why? While you cannot prove your primary hypothesis with 100% certainty (the closest you can get is 99%), you can disprove your null hypothesis. The null hypothesis is the opposite of your primary hypothesis. You can also have a secondary hypothesis, tertiary hypothesis, and so on. The most basic hypothesis test will involve an H0 (null hypothesis) and an H1 (your primary hypothesis). You can use these two experiments or analyses to identify causation within your product: Once you find a correlation, you can test for causation by running experiments that “ control the other variables and measure the difference.” Run robust experiments to determine causation For example, you might think you know which key activation event results in long-term user retention, but without rigorous testing, you risk making critical product decisions based on the wrong user behavior. A false positive can occur if you don’t extensively test the relationship between a dependent and an independent variable.įalse positives are problematic for product insights because you might incorrectly think you understand the link between important outcomes and user behaviors. It might be tempting to associate two variables as “cause and effect.” But doing so without confirming causality in a robust analysis can lead to a false positive-a causal relationship seems to exist but isn’t actually there. How to test for causation in your productĬausal relationships don’t happen by accident. In this example, joining communities and higher retention are correlated, but there could be a third factor causing both. Your analysis reveals a shocking finding: Users who joined at least one community have higher retention than those who did not join a community. One cohort only has users who joined a community, and the other only has users who did not join a community. You’re curious whether communities impact retention, so you create two equally-sized groups ( cohorts) with randomly selected users. You ask your team to develop a new feature that allows users to join “communities.”Ī month after you release your new communities feature, adoption sits at about 20% of all users. You hypothesize that customer retention for your product is linked to in-app social behaviors.

Picture this: You just launched a new version of your music-streaming mobile app. You might expect to find causality in your product, where specific user actions or behaviors result in a particular outcome. There is a chain reaction: A causes E, which leads E to cause B (but you only saw that A causes B from your own eyes).Īn example of correlation vs.There’s another variable involved: A does cause B-as long as D happens.The two are correlated, but there’s more to it: A and B are correlated, but they’re actually caused by C.The opposite is true: B actually causes A.Second, there are several other possibilities for an association, including: Why? First, our observations are purely anecdotal. However, we cannot simply assume causation even if we see two events happening, seemingly together, before our eyes. That would imply a cause-and-effect relationship, where one event is the result of another event. We often fabricate these explanations when two variables appear to be so closely associated that one is dependent on the other. Instead, both events are caused by something else-sunny weather.Ĭorrelation and causation are often confused because the human mind likes to find explanations for seemingly related events, even when they do not exist. But neither event actually causes the other. In this example, there’s a correlation between eating ice cream and getting sunburned because the two events are related.

Seen vs saw for dummies