Episode 94: Modern Metadata Management, Open-Source Adoption, and Early-Stage Culture with Mars Lan

Show Notes(01:41) Mars walked through his education studying Computer Systems Engineering at The University of Auckland in New Zealand.(03:16) Mars reflected on his overall Ph.D. experience in Computer Science at UCLA.(05:55) Mars discussed his early research paper on a robust and scalable lane departure warning system for smartphones.(07:13) Mars described his work on SmartFall, an automatic fall detection system to help prevent the elderly from falling.(08:34) Mars explained his project WANDA, an end-to-end remote health monitoring and analytics system designed for heart failure patients.(10:06) Mars recalled learnings from interning as a software engineer at Google during his Ph.D.(14:54) Mars discussed engineering challenges while working on PHP for Google App Engine and Gboard personalization during his subsequent four years at Google.(19:05) Mars rationalized his decision to join LinkedIn to lead an engineering team that builds the core metadata infrastructure for the entire organization.(21:15) Mars discussed the motivation behind the creation of LinkedIn’s generalized metadata search and discovery tool, DataHub, later open-sourced in 2020.(25:21) Mars dissected the key architecture of DataHub, which is designed to address the key scalability challenges coming in four different forms: modeling, ingestion, serving, and indexing.(28:50) Mars expressed the challenges of finding DataHub’s early adopters internally at LinkedIn and externally later on at other companies.(35:22) Mars shared the story behind the founding of Metaphor Data, which he co-founded with Pardhu Gunnam and Seyi Adebajo and currently serves as the CTO.(41:55) Mars unpacked how Metaphor’s modern metadata platform serves as a system of record for any organization’s data ecosystem.(48:07) Mars described new challenges with metadata management since the introduction of the modern data stack and key features of a great modern metadata platform (as brought up in his in-depth blog post with Ben Lorica).(53:55) Mars explained how a modern metadata platform fits within the broader data ecosystem.(58:30) Mars shared the hurdles to finding Metaphor Data’s early design partners and lighthouse customers.(01:04:33) Mars shared valuable hiring lessons to attract the right people who are excited about Metaphor’s mission.(01:07:28) Mars shared important culture-building lessons to build out a high-performing team at Metaphor.(01:10:45) Mars shared fundraising advice for founders currently seeking the right investors for their startups.(01:13:22) Closing segment.Mars’ Contact InfoTwitterLinkedInGoogle ScholarGitHubMetaphor DataWebsite | Twitter | LinkedInCareers | About PageData Documentation | Data CollaborationMentioned ContentArticlesDataHub: A generalized metadata search and discovery tool (Aug 2019)Open-sourcing DataHub: LinkedIn’s metadata search and discovery platform (Feb 2020)Founding Metaphor Data (Dec 2020)Metaphor and Soda partner to unify the modern data stack with trusted data (Dec 2021)Introducing Metaphor: The Modern Metadata Platform (Nov 2021)The Modern Metadata Platform: What, Why, and How? (Jan 2022)PapersSmartLDWS: A robust and scalable lane departure warning system for the smartphones (Oct 2009)SmartFall: An automatic fall detection system based on subsequence matching for the SmartCane (April 2009)WANDA: An end-to-end remote health monitoring and analytics system for heart failure patients (Oct 2012)PeopleBenn Stancil (Chief Analytics Officer at Mode Analytics, Well-Known Substack Writer)Tristan Handy (Co-Founder and CEO of dbt Labs, Writer of The Analytics Engineering Roundup)Andy Pavlo (Associate Professor of Database at Carnegie Mellon University)Books“Working In Public” (by Nadia Eghbal)“The Mom Test” (by Rob Fitzpatrick)“A Thousand Brains” (by Jeff Hawkins)“The Scout Mindset” (by Julia Galef)Notes

My conversation with Mars was recorded back in January 2022. Since then, many things have happened at Metaphor Data. I’d recommend:

Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.

Datacast is produced and edited by James Le. Get in touch with feedback or guest suggestions by emailing [email protected].

Subscribe by searching for Datacast wherever you get podcasts or click one of the links below:

If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.

Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.

Datacast is produced and edited by James Le. For inquiries about sponsoring the podcast, email [email protected].

Subscribe by searching for Datacast wherever you get podcasts, or click one of the links below:

If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.