Machine Learning Street Talk (MLST) artwork

#64 Prof. Gary Marcus 3.0

Machine Learning Street Talk (MLST)

English - February 24, 2022 15:44 - 51 minutes - 71.1 MB
Technology Homepage Download Google Podcasts Overcast Castro Pocket Casts RSS feed
Previous Episode: #063 - Prof. YOSHUA BENGIO - GFlowNets, Consciousness & Causality
Next Episode: #65 Prof. PEDRO DOMINGOS [Unplugged]

Patreon: https://www.patreon.com/mlst


Discord: https://discord.gg/HNnAwSduud


YT: https://www.youtube.com/watch?v=ZDY2nhkPZxw


We have a chat with Prof. Gary Marcus about everything which is currently top of mind for him, consciousness 




[00:00:00] Gary intro


[00:01:25] Slightly conscious


[00:24:59] Abstract, compositional models


[00:32:46] Spline theory of NNs


[00:36:17] Self driving cars / algebraic reasoning 


[00:39:43] Extrapolation


[00:44:15] Scaling laws


[00:49:50] Maximum likelihood estimation




References:


Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets


https://arxiv.org/abs/2201.02177




DEEP DOUBLE DESCENT: WHERE BIGGER MODELS AND MORE DATA HURT


https://arxiv.org/pdf/1912.02292.pdf




Bayesian Deep Learning and a Probabilistic Perspective of Generalization


https://arxiv.org/pdf/2002.08791.pdf