Decision first
Every measurement plan starts with the decision it has to inform, then works back to the evidence.
Measurement
Measure what the media changed.
Attribution guides the day, experiments show what the media caused, and the mix model sets the budget.
At the end of a quarter we lay the platforms' reports next to the orders the business actually booked. Each platform's number is honest about what it saw. Added together, they often claim more sales than there were.
That gap is where our measurement starts. Attribution tells you where conversions happened. It can't tell you what caused them, so we build measurement in layers and check each layer against the others.
A method for each question
Each method answers a different question. Used together and checked against each other, a practice called triangulation, they give one answer you can act on.
Attribution
Where did conversions happen? It guides daily optimization inside and across digital channels.
Experiments
What did the media cause? Holdouts, geo tests and matched markets compare people or places that saw the campaign with those that didn't.
Mix modeling
Where should the next budget go? A model of sales against spend in every channel, digital and traditional, with no tracking of individuals.
Calibration Experiment results calibrate the mix model. The model sets the budget, and attribution steers the day inside it.
Rules we agree before launch
Windows and metrics first
Attribution windows, success metrics and test design are agreed before the first dollar goes out.
Experiments over assumptions
Holdouts, geo tests and matched markets, sized before spend so the result means something.
Each sale counted once
Server-side, first-party events deduplicated against your actual orders, so each sale is counted once in our record, whatever each platform claims.
Every channel in the model
Streaming, linear TV, audio, out of home, print and mail sit in the same model as search and social. What we can't measure, we say.
When the numbers disagree
Platform-reported results are where we start, not where we stop. When attribution says one thing and an experiment says another, we show both, explain the gap and test again where the doubt sits.
How TV, audio, out of home and print are measured
Search, social and the open web are measured with clicks, conversions and lift tests, as set out on Digital advertising. Many channels never get a click. Each one leaves its own trace, and we measure it with that trace, then bring it into the mix model and geo tests with everything else.
Streaming and TV
Rating points, reach and frequency for linear TV, deduplicated with streaming where the data allows. Household impressions and completion from the servers that insert ads into streams, site visits right after an ad airs, and outcome studies.
Audio
Radio audiences from the measurement the medium is traded on, then geo tests, brand lift, promo codes and dedicated URLs, and impressions counted on each file request where ads are inserted dynamically.
Out of home
Audiences from the industry's audience measurement, as opportunity to see and likelihood to see, proof of play on digital screens, and exposed areas compared with control areas to read the lift.
Print and mail
Audited circulation, promo codes and dedicated addresses, matched markets where print runs region by region, buyers matched back against the mailing list, and control groups that receive nothing.
Retail and in store
Closed-loop sales from the retailer's data, with a stated model for the sales that can't be matched, such as cash or unrecognized cards.
Events and sponsorships
Codes, registrations, surveys, and test and control areas. The effect is hard to separate from the rest of the plan, so it is reported as an estimate.
What we call an estimate
Modeled store visits, out-of-home audiences and any channel left out of the model. Each one is labeled as an estimate in the plan and in the report, with the method behind it.
Questions we get
Which attribution model do you use?
The one your data can support, usually data-driven where there is enough volume, with windows as close as each platform allows for similar goals. We read it as a guide for daily decisions. Proof of cause comes from experiments.
What can't you measure?
Some effects are too small for the test a budget can afford, and some channels only report estimates. The plan and every report name them, with how the gap was handled.
How big does a test need to be?
Big enough to detect the smallest effect worth acting on, the minimum detectable effect. We size it before spend, from that effect, the noise in your sales and the time the test can run. If a budget can't support a clean test, we say so before it starts.
Media you can explain.
Bring a plan, a report or a question your numbers can't answer yet.