Multi-Touch Attribution — We Hardly Knew Ye
Marketing mix modeling (MMM) was not designed to measure ROI for small-scale, highly targeted digital campaigns. In 2014, multi-touch attribution (MTA) was the dream solution, tracking person-level media impressions, tying them to sales, and precisely measuring the value of every touchpoint. But in January 2015, Google starting limiting the release of person-level exposure data out of concern for “data leakage”, which really meant revenue leakage because sharing exposure data means giving away valuable targeting lists. Then Cambridge Analytica misappropriated data meant for academic purposes , targeting millions of Facebook users in the 2016 US election, costing Meta $725 million, and starting the current wave of privacy regulations. Today, from what began as self-interest and expanded to privacy concerns, an increasing number of major ad platforms don’t share person level exposure data.
Large-Scale Marketing RCT
Large-Scale Marketing RCTs can measure what marketing mix cannot: small-scale, highly targeted digital campaigns by specific target audience. Our tests indicate such RCTs can't practically measure all a brand's campaigns, but they can accurately measure 5-10 audiences per quarter, and behind most walled gardens. While large-scale RCTs can be used to measure virtually any attribute of a digital campaign, they're particularly helpful measuring audience quality as multiple audiences are generally run in lockstep causing collinearity issues that hamper marketing mix but do not affect RCT.
Large-scale RCTs address some major challenges encountered with MTA:
- Accurately measures specific target audiences
- Measures ROI for most walled garden campaigns
- Zero selection bias
- Minimal privacy issues
- Fast turnaround possible with initial results available Day 2 of the campaign
Our RCTs are GDPR / CCPA compliant and do not need person-level impression data, just person-level sales and person-level control groups. No tracking pixel is required.
Based on Established Science
RCT with a known control group but unknown subset of the treatment group who actually got the treatment is a common form of RCT called “intention to treat” (ITT), dating back to the 1960s. It addresses the fact that in clinical trials, the treatment group doesn't always follow the treatment. The same principle applies when we don't know exactly who saw an ad impression, just a control group who didn't. We can still accurately measure an average effect between the two groups and calculate ROI. And it's not just us saying that. Researchers from Google, Netflix, and Yahoo also use ITT RCT as the gold standard against which to evaluate other measurement approaches. See When Less is More: Data and Power in Advertising Experiments, Johnson et al, 2015 and Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness, Johnson et al, 2017. Because ITT RCT doesn't drop subjects for any reason, even if they didn't get the treatment, the researcher can't intentionally or unintentionally bias the results by dropping subjects. This near guarantee of zero bias is why the FDA mandates ITT RCT as the primary approach in all standard clinical trials. See Good Review Practice: Clinical Review of Investigational New Drug Applications, US Food and Drug Administration, 2013, pages 67-68.
Real-Time Measurement
I know we've called things real-time before but this time we really mean it. Our RCT results can start the day after the ad breaks. The day after the ad ends, we can report (near) final ROI. The main reason our RCTs can go so fast is we don't need to get exposure data from anybody. We create our own causal data when we create the test and control groups. So we have the causal data before the ad breaks.
