017 - John Cutler on Productizing Storytelling Measuring What Matters & Analytics Product Management

John Cutler is a Product Evangelist for Amplitude, an analytic platform that helps companies better understand users behavior, helping to grow their businesses. John focuses on user experience and evidence-driven product development by mixing and matching various methodologies to help teams deliver lasting outcomes for their customers. As a former UX researcher at AppFolio, a product manager at Zendesk, Pendo.io, AdKeeper and RichFX, a startup founder, and a product team coach, John has a perspective that spans individual roles, domains, and products.

In today’s episode, John and I discuss how productizing storytelling in analytics applications can be a powerful tool for moving analytics beyond vanity metrics. We also covered the importance of understanding customers’ jobs/tasks, involving cross-disciplinary teams when creating a product/service, and:

John and Amplitude’s North Star strategy and the (3) measurements they care about when tracking their own customers’ success
Why John loves the concept of analytics “notebooks” (also a particular feature of Amplitude’s product) vs. the standard dashboard method
Understanding relationships between metrics through “weekly learning users” who share digestible content
John’s opinions on involving domain experts and cross-discipline teams to enable products focused on outcomes over features
Recognizing whether your product/app is about explanatory or exploratory analytics
How Jazz relates to business – how you don’t know what you don’t know yet

Resources and Links:

Connect with John on LinkedIn

Follow John on Twitter

Keep up with John on Medium

Amplitude

Designing for Analytics

Quotes from Today’s Episode

“It’s like you know in your heart you should pair with domain experts and people who know the human problem out there and understand the decisions being made. I think organizationally, there’s a lot of organizational inertia that discourages that, unfortunately, and so you need to fight for it. My advice is to fight for it because you know that that’s important and you know that this is not just a pure data science problem or a pure analytics problem. There’s probably there’s a lot of surrounding information that you need to understand to be able to actually help the business.” – John

“We definitely ‘dogfood’ our product and we also ‘dogfood’ the advice we give our customers.” – John

“You know in your heart you should pair with domain experts and people who know the human problem out there and understand the decisions being made. […] there’s a lot of organizational inertia that discourages that, unfortunately, and so you need to fight for it. I guess my advice is, fight for it, because you know that it is important, and you know that this is not just a pure data science problem or a pure analytics problem.” – John

“It’s very easy to create assets and create code and things that look like progress. They mask themselves as progress and improvement, and they may not actually return any business value or customer value explicitly. We have to consciously know what the outcomes are that we want.” – Brian

“We got to get the right bodies in the room that know the right questions to ask. I can smell when the right questions aren’t being asked, and it’s so powerful” – Brian

“Instead of thinking about what are all the right stats to consider, [I sometimes suggest teams] write in plain English, like in prose format, what would be the value that we could possibly show in the data.’ maybe it can’t even technically be achieved today. But expressing the analytics in words like, ‘you should change this knob to seven instead of nine because we found out X, Y, and Z happened. We also think blah, blah, blah, blah, blah, and here is how we know that, and there’s your recommendation.’ This method is highly prescriptive, but it’s an exercise in thinking about the customer’s experience.” – Brian