Finding Ground Truth: Measuring & Testing Incrementality - Brian Krebs (MetricWorks)

For this episode our guest is Brian Krebs, the CEO of MetrickWorks, an expert on incrementality measurement and testing. Brian discusses how the use of incrementality has evolved to its present popularity within the mobile marketing industry and the confluence of issues that make it complex to test incremental lift. Brain says Mixed Media Modeling is the way to fill in the gaps with attribution, but it must be validated and calibrated with geo level ground truth experimentation. Find out why and how this works. 

MetricWorks is a third-party solution for measuring incremental lift. Their solution only uses aggregate data points impervious to privacy changes, making it “future proof.” MetricWorks also provides LTV predictions and full UA automation, driven by machine learning.

Questions Brian Answered in this Episode:What drives your entrepreneurial spirit?How has MetricWorks changed over the years and what is your focus today?How have you seen the evolution of incrementality measurement?What’s the process of working with MetricWorks to conduct an incremental lift study?Do you have a solution for apps that are only based in one country?Who stands to gain the most from incrementality measurement? What channels have been undervalued?Timestamp:2:45 Brian’s background7:20 About MetricWorks13:40 The evolution of incrementality15:55 Confluence of issues with testing incrementality19:05 Mixed Media Modeling: strengths and weaknesses20:05 MetricWorks’s incrementality measurement solution24:54 Ground truth incrementality testing28:26 Solutions for apps running in one country only33:33 What channels are being undervaluedQuotes:

(18:47-19:02) “Ghost bidding is just a magical solution. And not only is it really hurt by losing the device id, but as a marketer you’re finding yourself trusting the vendor. The vendor is necessary to do ghost bidding or ghost ads.”

(19:10-19:36) “[MMM] is complex but it’s simple in fundamental understanding of it, and, completely privacy compatible – doesn’t require any device-level data. And, there’s no need to get rid of incrementality testing. You have to go to a geo level, which is what we do. But, you can still use those signals to calibrate the MMM over time, which is probably MMM’s biggest weakness: not having some truth to ground in.”  

Mentioned in this Episode:Brian Kreb’s LinkedInEmail: [email protected]MetricWorks“Using MMM to Fill in the Gaps with Attribution”