Blog
Why Incrementality Testing is Important
Most marketers rely on Last-Click Attribution, which often credits a conversion to a branded search ad or a retargeting banner that the user would have clicked anyway. This leads to under value of upper funnel channels and overvalues lower funnel such as branded search.
Incrementality experiments answer the question of what would have happened with your marketing. Comparing KPI outcomes between exposed and controlled groups to isolate the impact of your marketing campaigns. Realize your measurement model from correlation to causal.
Methodologies
1) Audience Split (Randomized Control Trials)
Split a specific audience into two groups:
Test Group: Sees your ads.
Control (Holdout) Group: Sees no ads (or "Ghost Ads").
The Result: The difference in conversion rates between these two is your Incremental Lift.
Both Meta and Google have built in Conversion Lift studies to make running these tests easy, ensuring you will have a suffficient sample size and the confidence level of the outcomes.
2) Geo-Testing (The Modern Standard)
With user privacy controls increasing thereby limiting user level tracking, many brands use geographic (DMA) regions.
Example: Run ads in Ohio (Test), but turn them off in Michigan (Control), provided both markets have historically similar sales patterns.
Many marketers go with a "close enough" model of matched market tests or MMT. For example, how do Detroit and Baltimore compare in performance? This leads to a flawed strategy and inefficient spend. A better model is to stratification of audiences such as 70% Detroit, 20% Seattle, and 10% New Jersey. By using these synthetic controls this avoids the assumption that two markets are static and will change in the same way over time.
3) Media Mix Modeling (MMM)
Incrementality testing isn't just a one-off test, it should be integrated into long-term statistical models that analyze how different channels interact over time. Frequent testing improves accuracy of regression models and channel budget allocation.
4) Potential Experimental Roadmap
Potential tests include:
- Broad vs Interest targeting
- Branded vs Non-Branded Keywords
- UGC vs Informational Videos
- Call to Action
- Click vs View Optimization
