Glossary · Analytics & Tracking

Incrementality Testing

Definition

Incrementality testing measures how many sales a marketing activity actually caused, by comparing a group exposed to it with a similar group that was not.

Also called: Lift testing, holdout testing

Attribution decides which touchpoint gets credit for a sale. Incrementality asks a harder question: would the sale have happened anyway? To answer it, you compare a test group that sees the marketing with a control group that doesn’t, and the difference in outcomes is the incremental lift.

Common methods for eCommerce brands include:
– Audience holdouts: randomly exclude a share of an audience from ads, or from an email or SMS flow, and compare purchases.
– Geo tests: turn spend on or off in matched regions and compare sales by region.
– Platform lift studies: Meta and Google both offer lift studies for eligible advertisers.

Why it matters for eCommerce

Retargeting, branded search and emails to loyal customers often receive credit for orders those customers would have placed regardless. Incrementality testing shows which spend actually creates new profit, so you can move budget away from activity that only harvests existing demand and toward activity that grows it.

How it works in practice

  • Start with one clear question, such as “Does retargeting drive orders we would not otherwise get?”
  • Decide the primary metric (new customers, orders or contribution margin) and the test length before launch.
  • Keep the control group truly unexposed, and match test regions by size and seasonality.
  • Translate lift into incremental contribution margin per dollar spent, then compare it with platform-reported results.

Common mistakes

  • Running tests too small or too short to detect a real difference.
  • Testing during a major promotion or alongside other big changes that contaminate the result.
  • Treating one result as permanent. Channels, creative and audiences change, so retest your biggest channels periodically.

Put the numbers to work

Our free calculators turn these definitions into your numbers, and the Free Profit Audit shows which ones cost you the most.

Free Profit Audit