Episode 61: Meta Reinforcement Learning with Louis Kirsch

Show Notes(2:05) Louis went over his childhood as a self-taught programmer and his early days in school as a freelance developer.(4:22) Louis described his overall undergraduate experience getting a Bachelor’s degree in IT Systems Engineering from Hasso Plattner Institute, a highly-ranked computer science university in Germany.(6:10) Louis dissected his Bachelor thesis at HPI called “Differentiable Convolutional Neural Network Architectures for Time Series Classification,” — which addresses the problem of automatically designing architectures for time series classification efficiently, using a regularization technique for ConvNet that enables joint training of network weights and architecture through back-propagation.(7:40) Louis provided a brief overview of his publication “Transfer Learning for Speech Recognition on a Budget,” — which explores Automatic Speech Recognition training by model adaptation under constrained GPU memory, throughput, and training data.(10:31) Louis described his one-year Master of Research degree in Computational Statistics and Machine Learning at the University College London supervised by David Barber.(12:13) Louis unpacked his paper “Modular Networks: Learning to Decompose Neural Computation,” published at NeurIPS 2018 — which proposes a training algorithm that flexibly chooses neural modules based on the processed data.(15:13) Louis briefly reviewed his technical report, “Scaling Neural Networks Through Sparsity,” which discusses near-term and long-term solutions to handle sparsity between neural layers.(18:30) Louis mentioned his report, “Characteristics of Machine Learning Research with Impact,” which explores questions such as how to measure research impact and what questions the machine learning community should focus on to maximize impact.(21:16) Louis explained his report, “Contemporary Challenges in Artificial Intelligence,” which covers lifelong learning, scalability, generalization, self-referential algorithms, and benchmarks.(23:16) Louis talked about his motivation to start a blog and discussed his two-part blog series on intelligence theories (part 1 on universal AI and part 2 on active inference).(27:46) Louis described his decision to pursue a Ph.D. at the Swiss AI Lab IDSIA in Lugano, Switzerland, where he has been working on Meta Reinforcement Learning agents with Jürgen Schmidhuber.(30:06) Louis created a very extensive map of reinforcement learning in 2019 that outlines the goal, methods, and challenges associated with the RL domain.(33:50) Louis unpacked his blog post reflecting on his experience at NeurIPS 2018 and providing updates on the AGI roadmap regarding topics such as scalability, continual learning, meta-learning, and benchmarks.(37:04) Louis dissected his ICLR 2020 paper “Improving Generalization in Meta Reinforcement Learning using Learned Objectives,” which introduces a novel algorithm called MetaGenRL, inspired by biological evolution.(44:03) Louis elaborated on his publication “Meta-Learning Backpropagation And Improving It,” which introduces the Variable Shared Meta-Learning framework that unifies existing meta-learning approaches and demonstrates that simple weight-sharing and sparsity in a network are sufficient to express powerful learning algorithms.(51:14) Louis expands on his idea to bootstrap AI that entails how the task, the general meta learner, and the unsupervised objective should interact (proposed at the end of his invited talk at NeurIPS 2020).(54:14) Louis shared his advice for individuals who want to make a dent in AI research.(56:05) Louis shared his three most useful productivity tips.(58:36) Closing segment.Louis’s Contact InfoWebsiteTwitterLinkedInGoogle ScholarGitHubMentioned Content

Papers and Reports

Differentiable Convolutional Neural Network Architectures for Time Series Classification (2017)Transfer Learning for Speech Recognition on a Budget (2017)Modular Networks: Learning to Decompose Neural Computation (2018)Contemporary Challenges in Artificial Intelligence (2018)Characteristics of Machine Learning Research with Impact (2018)Scaling Neural Networks Through Sparsity (2018)Improving Generalization in Meta Reinforcement Learning using Learned Objectives (2019)Meta-Learning Backpropagation And Improving It (2020)

Blog Posts

Theories of Intelligence — Part 1 and Part 2 (July 2018)Modular Networks: Learning to Decompose Neural Computation (May 2018)How to Make Your ML Research More Impactful (Dec 2018)A Map of Reinforcement Learning (Jan 2019)NeurIPS 2018, Updates on the AI Roadmap (Jan 2019)MetaGenRL: Improving Generalization in Meta Reinforcement Learning (Oct 2019)General Meta-Learning and Variable Sharing (Nov 2020)

People

Jeff Clune (for his push on meta-learning research)Kenneth Stanley (for his deep thoughts on open-ended learning)Jürgen Schmidhuber (for being a visionary scientist)

Book

“Grit” (by Angela Duckworth)

About the show

Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.

Datacast is produced and edited by James Le. For inquiries about sponsoring the podcast, email [email protected].

Subscribe by searching for Datacast wherever you get podcasts, or click one of the links below:

Listen on SpotifyListen on Apple PodcastsListen on Google Podcasts

If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.