061: Interpolation, Extrapolation and Linearisation (Prof. Yann LeCun, Dr. Randall Balestriero)

Machine Learning Street Talk (MLST)

English - January 04, 2022 12:59 - 3 hours - 183 MB
Technology Homepage Download Google Podcasts Overcast Castro Pocket Casts RSS feed

Previous Episode: #60 Geometric Deep Learning Blueprint (Special Edition)

Next Episode: #062 - Dr. Guy Emerson - Linguistics, Distributional Semantics

We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST

Patreon: https://www.patreon.com/mlst

Yann LeCun thinks that it's specious to say neural network models are interpolating because in high dimensions, everything is extrapolation. Recently Dr. Randall Balestriero, Dr. Jerome Pesente and prof. Yann LeCun released their paper learning in high dimensions always amounts to extrapolation. This discussion has completely changed how we think about neural networks and their behaviour.

[00:00:00] Pre-intro

[00:11:58] Intro Part 1: On linearisation in NNs

[00:28:17] Intro Part 2: On interpolation in NNs

[00:47:45] Intro Part 3: On the curse

[00:48:19] LeCun

[01:40:51] Randall B

YouTube version: https://youtu.be/86ib0sfdFtw