28. Simpson’s Paradox
Estimated Reading Time: 1 Minute
Welcome to Aha Moments - The 1-minute newsletter with data and behavioral science insights for busy marketers like you.
Look at the results of a new marketing campaign across different customer segments:
• Segment A (younger customers): High engagement with the campaign.
• Segment B (older customers): Low engagement with the campaign.
• Overall (Segment A+B): campaign engagement appears low.
So, is the campaign effective?
Here is another one - A/B test results for a new website layout show:
• Mobile users: Version A of the website performs better.
• Desktop users: Version A of the website performs better.
• Overall(Mobile+ Desktop), Version B of the website performs better.
So which version is better?
The above two data sets are examples of a statistical phenomenon called Simpson’s Paradox, in which the association between two variables is reversed when a third variable is introduced.
Simpson’s Paradox occurs when a particular trend is seen in different data groups but reverses or disappears when combined.
If you only look at the overall data, you may conclude that the marketing campaign is ineffective and discontinue it. However, examining the segmented data shows that the campaign is very effective for younger customers. This insight allows you to refine the campaign targeting strategy.
Similarly, Simpson’s Paradox can mislead us into thinking Version B is the better option. However, when breaking down the data by device type, we see that Version A performs better for mobile and desktop users. Next, we need to investigate why the combined data shows a different trend. It could be due to traffic volume differences between mobile and desktop users.
Simpson’s Paradox is an essential data science concept for marketers to understand as it shows the importance of segmenting data and understanding the underlying patterns within different customer groups.