Josh Wills - Statistician, Former Director of Data Engineering at Slack #51

Welcome to another episode of Develomentor. Today's guest is Josh Wills. 

Josh Wills is a former engineer on Slack's Search, Learning, and Intelligence Team helping to build the company's production search and machine learning infrastructure. He's a recovering manager, having most recently built and led Slack's data engineering team and before that the data science and engineering teams at Cloudera and Google. 

Josh is a member of the Apache Software Foundation, the founder of the Apache Crunch project, and a co-author of O'Reilly's Advanced Analytics with Spark. In May of 2012, he tweeted a pithy definition of a data scientist as someone who is better at statistics than any software engineer and better at software engineering than any statistician, and his Twitter mentions have never been the same.

Click Here –> For more information about tech careers

Episode Summary

“Statistics in school, at least when I was there, was very much focused on the problem. 'How do we extract as much information as we can from a relatively small subset of data.' Statistics at Google and Facebook was much more concerned with ‘we are drowning in data, we have an overwhelming amount of data. How do we not delude ourselves?'"

"If you want to be a mentee, there's no work that you're too good to do. The best learning opportunities of my life came from situations where I was more than willing to roll up my sleeves and do things that are not sexy and are not cool."

“I didn’t go back to engineering. I never stopped engineering. I was always engineering while I was doing this management stuff on the side. Recruiting and hiring, I enjoy. And I very much enjoy mentoring people. But I do not personally care about or tolerate sitting through a sprint review or doing a budget planning meeting or, god help me, sitting through a promotion process.”

—Josh Wills

In this episode we’ll cover:

What are the differences in studying statistics in school versus working as a statistician at Google or Facebook?Why Josh chose to be a high-level individual contributor (IC) over management.How Josh worked a full-time job at IBM while getting his master's degree.

Additional Resources

The Manager's Path - by Camille Fournier
Software 2.0 by Andrej Karpathy - https://medium.com/@karpathy/software-2-0-a64152b37c35
How to Take Smart Notes - by Sönke Ahrens
Josh's talk how to play well with others about mistakes he has made during his career

You can find more resources in the show notes
To learn more about our podcast go to https://develomentor.com/
To listen to previous episodes go to https://develomentor.com/blog/

Follow Josh Wills
Twitter: @josh_wills
LinkedIn: linkedin.com/in/josh-wills-13882b/

Follow Develomentor:
Twitter: @develomentor

Follow Grant Ingersoll
Twitter: @gsingers
LinkedIn:

Support the Show.