Some customers who click an ad would have bought anyway. Retargeting ads, in particular, often reach people who were already about to purchase. Attribution reports credit the ad for those sales; incrementality tests reveal how many were genuinely caused by it. The difference can change budget decisions dramatically.
The core idea
Split a similar audience into two groups. One group sees the campaign (the test group); the other does not (the control or holdout group). Compare results. The difference between the groups is the incremental lift. The Academy's glossary explains incrementality.
Types of incrementality test
| Test | How it works | Best for |
|---|---|---|
| Conversion lift | Platform randomly holds back ads from a share of the audience | Paid social campaigns on Meta and others |
| Geo test | Run ads in some regions and not in comparable others | Businesses with regional sales data |
| Time-based test | Pause a channel for a period and compare | Simple checks, with caution |
| Brand lift | Survey exposed and unexposed groups | Awareness campaigns |
The Academy's guide to a brand lift study covers awareness measurement.
Platform lift tests
Meta offers conversion lift studies for eligible advertisers, and other platforms offer similar tools, often requiring minimum budgets. The platform randomly assigns people to test and control groups and reports the incremental conversions. Availability and requirements change, so check current options in each ad platform.
Geo tests
Choose regions that behave similarly, such as two cities with comparable past sales. Run ads in one and not the other, then compare sales. Geo tests work with any sales data, including offline sales, but need careful region selection and enough volume.
Reading results
- Calculate lift: test conversions minus control conversions, adjusted for group size.
- Check significance: is the difference larger than random variation?
- Calculate incremental cost per acquisition: spend divided by incremental conversions.
- Compare with attribution to see how much reported conversions overstate or understate impact.
When incrementality testing makes sense
- Significant ad spend, where small improvements matter.
- Retargeting campaigns, which often overstate impact.
- Budget decisions between channels.
- Doubts about attribution numbers.
For small budgets, tests may not have enough data to give reliable answers. In that case, simpler signals such as customer surveys and before-and-after comparisons are a reasonable substitute, as long as their limits are acknowledged.
Organic social and incrementality
Organic content is harder to test because you cannot easily hold back an audience. Some businesses pause a platform or a format for a period, or compare regions with different activity, but results need careful interpretation. Organic reach depends on platform signals; TikTok's documented signals are completion rate and rewatches, shares, comments, likes, interest signals from watch behaviour✓, so organic effects vary naturally week to week.
Incrementality versus attribution
| Attribution | Incrementality | |
|---|---|---|
| Question | Which touchpoints did buyers pass through? | What did marketing cause? |
| Method | Tracking and rules | Controlled comparison |
| Frequency | Continuous | Periodic tests |
| Weakness | Credits sales that would have happened anyway | Needs volume and planning |
The Academy's guide to attribution models explains the attribution side.
Designing a simple test
Choose one campaign, one outcome and a test length long enough to capture purchase cycles, often two to four weeks. Keep everything else stable during the test. Write down your hypothesis and what result would change your decision before you start.
Acting on results
If a campaign shows little incremental lift, reduce spend or change targeting and creative. If lift is strong, consider increasing investment. Repeat tests periodically, as audiences and platforms change.
Explaining results to stakeholders
Incrementality results can be uncomfortable, especially when a favourite campaign shows little lift. Present them calmly, with the method, the numbers and the uncertainty. Frame the outcome as an opportunity to move budget to where it works, rather than a failure. Leadership tends to welcome evidence that improves returns, even when it overturns earlier reports.
A worked example
An online shop spends £3,000 a month on retargeting ads. Meta reports 150 purchases attributed to the campaign. A conversion lift test holds back ads from a random tenth of the audience. Purchases in the held-back group are only slightly lower than in the exposed group, and the platform estimates 40 incremental purchases. The true cost per incremental purchase is £75, not the £20 the attribution report suggested. The shop moves some budget from retargeting to prospecting campaigns that reach new customers.
Common objections
Some teams worry that holding back ads loses sales. In practice, the control group is usually small, and the knowledge gained can save far more than the sales risked. Others worry that tests are too complex. Platform lift tools handle much of the setup, and a simple geo test can be run with a spreadsheet. The main requirement is patience and a willingness to accept results that may challenge assumptions.
Incrementality and creative
Tests can also compare creative approaches, such as video versus static images, by measuring which produces more incremental conversions rather than more attributed clicks. This is especially useful when attribution favours one format for reasons that have little to do with its real impact.
Common mistakes
- Treating attributed conversions as incremental.
- Running tests without enough volume.
- Changing other variables during a test.
- Stopping tests too early.
- Never repeating tests.
Learning advanced measurement
The Academy's advanced measurement module in the fourth level covers incrementality, lift studies and modelling, and the paid social module covers campaign structure. The free assessment shows where you would start. The Academy's guide to marketing mix modelling covers a complementary approach.
