Solutions
Randomized Controlled Trials
Randomized controlled trials were developed in the medical industry to eliminate selection bias in clinical trials. In some industries and markets, we offer either market-level or large-scale person-level RCTs with millions of people, enabling us to measure specific target audience quality, something marketing mix can't practically do as marketers typically target several audience segments at once. The technique we use requires knowing who did not see an ad, but not who did, enabling us to measure most walled gardens with person-level data.
01
Multi-Touch Attribution — We Hardly Knew Ye
- 2014
- In 2014, multi-touch attribution (MTA) was the dream solution, tracking person-level media impressions, tying them to sales, and precisely measuring the value of every touchpoint.
- January 2015
- But in January 2015, Google started limiting the release of person-level exposure data out of concern for “data leakage”, which really meant revenue leakage because sharing exposure data means giving away valuable targeting lists.
- 2016
- Then Cambridge Analytica misappropriated data meant for academic purposes, targeting millions of Facebook users in the 2016 US election, costing Meta $725 million, and starting the current wave of privacy regulations.
- Today
- Today, from what began as self-interest and expanded to privacy concerns, an increasing number of major ad platforms don’t share person-level exposure data.
02
Large-Scale Marketing RCT
Large-Scale Marketing RCTs can measure what marketing mix cannot: small-scale, highly targeted digital campaigns by specific target audience.
- 5–10
- Audiences measured accurately per quarter
- Day 2
- Of the campaign, initial results available
- 0
- Tracking pixels required
Our tests indicate such RCTs can't practically measure all a brand's campaigns, but they can accurately measure 5-10 audiences per quarter, and behind most walled gardens.
While large-scale RCTs can be used to measure virtually any attribute of a digital campaign, they're particularly helpful for measuring audience quality as multiple audiences are generally run in lockstep causing collinearity issues that hamper marketing mix but do not affect RCT.
Large-scale RCTs address some major challenges encountered with MTA:
- Accurately measures specific target audiences
- Measures ROI for most walled garden campaigns
- Zero selection bias
- Minimal privacy issues
- Fast turnaround possible with initial results available Day 2 of the campaign

- Privacy
- Our RCTs are GDPR / CCPA compliant and do not need person-level impression data, just person-level sales and person-level control groups. No tracking pixel is required.
03
Based on Established Science
RCT with a known control group but unknown subset of the treatment group who actually got the treatment is a common form of RCT called “intention to treat” (ITT), dating back to the 1960s.
It addresses the fact that in clinical trials, the treatment group doesn't always follow the treatment. The same principle applies when we don't know exactly who saw an ad impression, just a control group who didn't. We can still accurately measure an average effect between the two groups and calculate ROI.
And it's not just us saying that. Researchers from Google, Netflix, and Yahoo also use ITT RCT as the gold standard against which to evaluate other measurement approaches [1, 2].
Because ITT RCT doesn't drop subjects for any reason, even if they didn't get the treatment, the researcher can't intentionally or unintentionally bias the results by dropping subjects. This near guarantee of zero bias is why the FDA mandates ITT RCT as the primary approach in all standard clinical trials [3].
References
- [1]When Less is More: Data and Power in Advertising Experiments, Johnson et al., 2015.
- [2]Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness, Johnson et al., 2017.
- [3]Good Review Practice: Clinical Review of Investigational New Drug Applications, US Food and Drug Administration, 2013, pages 67-68.
04
Real-Time Measurement
I know we've called things real-time before but this time we really mean it.
01 · Before the ad breaks
We create our own causal data when we create the test and control groups. So we have the causal data before the ad breaks.
02 · Day after the ad breaks
Our RCT results can start the day after the ad breaks.
03 · Day after the ad ends
The day after the ad ends, we can report (near) final ROI.
The main reason our RCTs can go so fast is we don't need to get exposure data from anybody.
