Estimating spillover between exposed and control audiences

Estimate the spillover your records can observe: compare assignment with delivery and outcome location, then state how much exposure remains unobserved. An impression count is not a count of contaminated people, and neither measure automatically tells you how much the lift estimate is biased.

Define the intended separation

In a user holdout, identify the campaigns withheld from control-assigned users. Exposure to those same tested campaigns through another route can breach the intended separation. Exposure to other, untested campaigns is background activity unless it changes the contrast the study was meant to measure.

In a geographic test, media may reach control areas, or a person may see an ad in a treatment area and later produce an outcome recorded in a control area. Those mechanisms require different data and denominators. Document the assignment rule, delivery locations and outcome-location rule before calculating a leakage measure.

Report observable leakage precisely

Where lawful and technically available, compare control assignment with delivery records for the tested campaign. State the number or share of control-assigned people with observed exposure, the observation window and how many have unknown exposure status. An aggregate platform lift report alone may not provide the person-level data needed for that calculation.

For a geographic test, reconcile delivery by assigned area and date. The share of recorded test-campaign impressions delivered in control areas is an impression leakage measure. It is not the share of control customers exposed: people can receive different numbers of impressions, and some exposure may be outside the records.

Keep travel, broad-coverage media and outcomes recorded away from exposure locations as separate mechanisms.

Check budgets as well as targeting. In a budget-constrained go-dark setup, restricting areas can shift spend into the remaining areas and alter the planned comparison even if no impression is mislocated.

Show what the result can still mean

Present the planned lift analysis with observed breaches and missing-data coverage. If plausible contamination rates can be bounded, a sensitivity analysis can show whether the decision changes under stated assumptions. A universal multiplier cannot reliably correct lift for travel, unobserved exposure and redirected budgets.

When the achieved media contrast differs from the plan, describe that difference. The result may reflect assignment as implemented; it does not automatically estimate the causal effect of actually seeing an ad.

If leakage makes the intended question unanswerable, consider different areas, tighter exclusions or another assignment unit for the next test. Check that enough units and outcome volume remain.