Multi-Touch Attribution – We Hardly Knew Ye
Marketing mix modeling (MMM) was not designed to measure ROI for small-scale, targeted digital campaigns. In 2014, multi-touch attribution (MTA) was the dream solution, tracking person-level media impressions, tying them to sales, and precisely measuring the value of every touchpoint. Then the walls went up. In January 2015, concerned about data (revenue) leakage, Google restricted tracking pixels on their display ad network. In 2016, Cambridge Analytica improperly used Facebook data, sparking a deep dive into digital data privacy. Today, from what began as self-interest and expanded to privacy concerns, the two largest walled gardens – Google and Facebook – representing over half of US digital media spend, have largely shut off third party access to their person-level impression data.
Large-Scale Marketing RCT
Large-Scale Marketing RCTs can measure what marketing mix cannot: individual, detailed digital campaigns. Our tests indicate such RCTs can't practically measure everything marketing mix can, but they can accurately measure 5-10 detailed campaigns per quarter per brand, even though those campaigns may be highly targeted, highly correlated, and behind walled gardens. Our RCTs use a random non-exposed control group. While they can be used to measure virtually any attribute of a digital campaign, ROI by target audience / segment is the sweet spot, because that's a big hole for marketing mix. Large-scale RCTs address some major challenges we encountered with MTA:
- Accurately measures small digital campaigns and target audiences
- Measures ROI for most walled garden campaigns
- Zero selection bias
- Minimal privacy issues
- Fast turnaround with initial results available Day 2 of the campaign
Our RCTs are GDPR / CCPA compliant and do not need person-level impression data, just person-level sales and person-level control groups. No tracking pixel is required.
Based on Established Science
RCT with a known control group but unknown subset of the treatment group who actually got the treatment is a common form of RCT called “intention to treat” (ITT), dating back to the 1960s. It addresses the fact that in clinical trials, the treatment group doesn't always follow the treatment. The same principle applies when we don't know exactly who saw an ad impression, just a control group who didn't. We can still accurately measure an average effect between the two groups and calculate ROI. And it's not just us saying that. Researchers from Google, Netflix, and Yahoo also use ITT RCT as the gold standard against which to evaluate other measurement approaches. See When Less is More: Data and Power in Advertising Experiments, Johnson et al, 2015 and Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness, Johnson et al, 2017. Because ITT RCT doesn't drop subjects for any reason, even if they didn't get the treatment, the researcher can't intentionally or unintentionally bias the results by dropping subjects. This near guarantee of zero bias is why the FDA mandates ITT RCT as the primary approach in all standard clinical trials. See Good Review Practice: Clinical Review of Investigational New Drug Applications, US Food and Drug Administration, 2013, pages 67-68.
Real-Time Measurement
I know we've called things real-time before but this time we really mean it. Our RCT results start the day after the ad breaks. The day after the ad ends, we report final ROI. Part of why we can go so fast is we get sales data from Catalina 3am every morning for the previous day. But a bigger reason is we don't need to get exposure data from anybody. We create our own causal data when we create the test and control groups. So we have the causal data before the ad begins.
