Measurement Design
Part of Media measurement planning
Planning measurement before money is committed: checks
Use a pre-approval measurement gate to confirm the outcome, method, data access, timing and limits before media is booked.
Before approving media investment, tie the evidence to the decision: name the business outcome, proposed method, data path, access owner and date the result is needed. Approve only when the method can measure that outcome, the necessary records are accessible and the evidence will arrive in time to use.
Set a measurement gate at approval
Write one sentence naming the business outcome and the decision it will inform. Work through the evidence requirements with the people responsible for media, analytics and the business record:
- Outcome:What exact event counts, and who can extract it?
- Coverage:Which Australian markets, products, channels and dates are included?
- Method:Will the report describe delivery, allocate credit to actions, or estimate an effect against a comparison?
- Access:Will the team receive the required supplier data, event definitions and reports?
- Timing:When will the data be available, and is that soon enough for the decision?
- Limit:What will remain unknown even if collection works as planned?
First, identify whether the claim is about describing delivery, allocating credit or estimating an effect. Attribution models are relevant to credit allocation; controlled experiments are used to measure advertising impact. Choose a method that supports the claim rather than treating these as interchangeable.
IAB Australia identifies controlled experiments as central to measuring digital advertising effectiveness and says they measure incrementality, not correlation. Its guide includes geo-tests and Test & Control surveys among the approaches used to assess impact.
For an effect claim, confirm the proposed comparison before exposure begins. Check that the sample size in each cell is adequate for the precision needed, that test and control groups are assigned randomly and match on relevant characteristics, and that each subject is in only one group. Smaller differences require larger samples.
If the proposal cannot show that its sample can support the required precision, narrow the claim or hold approval until a feasible method is set. A chart assembled later cannot, by itself, establish an experimental comparison.
Pre-approval measurement checks
- OutcomeWhat exact event counts, and who can extract it?
- CoverageWhich Australian markets, products, channels and dates are included?
- MethodWill the report describe delivery, allocate credit to actions, or estimate an effect against a comparison?
- AccessWill the team receive the required supplier data, event definitions and reports?
- TimingWhen will the data be available, and is that soon enough for the decision?
- LimitWhat will remain unknown even if collection works as planned?
Measuring delivery vs. allocating credit vs. estimating effect
- Describing deliveryVerifies ad impressions, clicks or views; used for reporting compliance.
- Allocating creditUses attribution models to assign conversion value across touchpoints; may not reflect incrementality.
- Estimating effectRequires controlled experiments (e.g. geo-tests, Test & Control surveys) to measure incrementality; must define comparison before exposure.
Key measurement requirements for experimental validity
- Sample size adequacy
- Must support required precision; smaller differences need larger samples
- Random assignment
- Test and control groups must be assigned randomly
- Matching characteristics
- Groups must match on relevant variables (e.g. demographics, behaviour)
- Single group membership
- Each subject must be in only one group (test or control)
Check the data path
Trace the proposed customer action from the placement to the destination, recorded event and business system. Check whether cancellations, repeat submissions and offline completions can be identified, so the counted outcome matches the business definition.
Confirm who can provide and access the exposure and outcome records, and when access will be available. An event in an ad account alone does not establish that it matches an accepted order or lead.
For a Test & Control survey, confirm how exposed respondents will be identified and how control respondents will be recruited. IAB Australia's guide describes controls collected at random from visitors to the same sites who have not seen the campaign creative.
If the design uses tracking technology or personal information, arrange a privacy review of the actual collection and sharing arrangements. For tracking pixels, consult the OAIC guidance titled “Tracking pixels and privacy obligations”.
Include the evidence in the commitment
Confirm when each required record or report will be available and when the result can be delivered. For a Test & Control survey, the test and control respondents should be collected simultaneously throughout the campaign period; check that the result will arrive before the decision it is meant to inform.
Measurement may require analytics work, survey fieldwork, supplier reporting, data access or internal time. Put those requirements and deadlines beside the proposed media commitment, and decide what to do if access is declined or the method proves infeasible.
Before approval, document the outcome definition, method, feasible data path, access arrangements, reporting owner and limits of the proposed result. Approve only when that evidence supports the intended claim and can be used in time; otherwise narrow the claim or hold approval.
After booking, keep a dated record of changes that affect comparability, such as a revised conversion rule or delayed launch.
