Episode 120: Next-Generation Experimentation, Statistics Engineering, and The Modern Growth Stack with Chetan Sharma

Show Notes(01:44) Chetan reflected on his undergraduate experience at Stanford studying Electrical Engineering and Statistics back in the late 2000s.(06:10) Chetan recalled his experience interning at IBM and Quantcast and doing research at Stanford Center for Minds, Brain, and Computation.(08:41) Chetan talked about his first job working as a research analyst focused on healthcare policy at Acumen.(11:15) Chetan walked through his decision to join Airbnb as their 4th data scientist and work on building Airbnb's original ETL framework for online risk mitigation.(15:12) Chetan recalled the early state of data science at Airbnb.(18:17) Chetan touched on the development of Airbnb's knowledge management and sharing platform called Knowledge Repo.(23:10) Chetan explained why an experimentation program is the most impactful thing a data team can do.(26:06) Chetan walked through the evolution of Airbnb's experimentation platform since its inception in 2014.(31:24) Chetan recalled fond memories from taking a year off from work to travel.(35:16) Chetan touched on his transition back to work by way of living in Atlanta and co-founding a logistics software startup called Saltbox.(39:28) Chetan described his time as a data scientist at Webflow, building their experimentation system from scratch.(42:48) Chetan shared the story behind the founding of Eppo.(46:12) Chetan dissected the key capabilities that are baked into the Eppo product.(48:32) Chetan dived deeper into the problems caused by long experiment durations and the benefits of using CUPED to bend time in experiments.(52:04) Chetan talked about the role of a statistics engineer.(54:45) Chetan shared his perspective on the role of experimentation in the Modern Data Stack and the Modern Growth Stack.(01:00:47) Chetan discussed the core elements of the modern experimentation stack.(01:04:54) Chetan talked about the experiment overhead.(01:06:24) Chetan emphasized the designer gap in experimentation tools(01:08:50) Chetan shared his thoughts about metric strategy.(01:10:51) Chetan shared valuable hiring lessons to attract the right people who are excited about Eppo's mission.(01:14:10) Chetan provided his perspectives on finding design partners for an early-stage startup.(01:17:04) Chetan shared fundraising advice to founders who are seeking the right investors for their startups.(01:19:40) Closing segment.Chetan's Contact InfoLinkedInTwitterGitHubAngelListEppo's ResourcesWebsite | Twitter | LinkedInBlog | Updates | DocAbout | CareersExperimentation ProductFeature Flagging ProductMentioned ContentArticlesTravel Year Facts and Superlatives (Dec 2019)Why I Started Eppo (Feb 2021)Reducing Experiment Durations (June 2021)The Designer Gap in Experimentation Tools (June 2021)We're Hiring A Statistics Engineer! (Aug 2021)Should You Always Run An Experiment? (Aug 2021)Stop Micromanaging Product Strategy (Sep 2021)The most impactful thing a data team can do is establish an experimentation program (Dec 2021)Bending Time in Experimentation (June 2022)We Raised $19.5M! (June 2022)Experimentation for the Modern Growth Stack: Our Investment in Eppo (June 2022)PeopleMike KaminskySean TaylorJeremy HowardBookThe Mom's Test (by Rob Fitzpatrick)Notes

My conversation with Chetan was recorded back in August 2022. Since then, Eppo has launched feature flagging, and now offers the first "flags on top of your warehouse" experimentation platform. They also have Miro, Twitch, DraftKings, and Zapier as customers.

Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.

Datacast is produced and edited by James Le. For inquiries about sponsoring the podcast, email [email protected].

Subscribe by searching for Datacast wherever you get podcasts, or click one of the links below:

If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.

Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.

Datacast is produced and edited by James Le. For inquiries about sponsoring the podcast, email [email protected].

Subscribe by searching for Datacast wherever you get podcasts, or click one of the links below:

If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.