Machine Learning Street Talk (MLST)
156 episodes - English - Latest episode: 5 days ago -Welcome! We engage in fascinating discussions with pre-eminent figures in the AI field. Our flagship show covers current affairs in AI, cognitive science, neuroscience and philosophy of mind with in-depth analysis. Our approach is unrivalled in terms of scope and rigour – we believe in intellectual diversity in AI, and we touch on all of the main ideas in the field with the hype surgically removed. MLST is run by Tim Scarfe, Ph.D (https://www.linkedin.com/in/ecsquizor/) and features regular appearances from MIT Doctor of Philosophy Keith Duggar (https://www.linkedin.com/in/dr-keith-duggar/).
Homepage Google Podcasts Overcast Castro Pocket Casts RSS feed
Episodes
#54 Gary Marcus and Luis Lamb - Neurosymbolic models
June 04, 2021 08:41 - 2 hours - 132 MBProfessor Gary Marcus is a scientist, best-selling author, and entrepreneur. He is Founder and CEO of Robust.AI, and was Founder and CEO of Geometric Intelligence, a machine learning company acquired by Uber in 2016. Gary said in his recent next decade paper that — without us, or other creatures like us, the world would continue to exist, but it would not be described, distilled, or understood. Human lives are filled with abstraction and causal description. This is so powerful. Francois Cho...
#53 Quantum Natural Language Processing - Prof. Bob Coecke (Oxford)
May 19, 2021 11:07 - 2 hours - 189 MBBob Coercke is a celebrated physicist, he's been a Physics and Quantum professor at Oxford University for the last 20 years. He is particularly interested in Structure which is to say, Logic, Order, and Category Theory. He is well known for work involving compositional distributional models of natural language meaning and he is also fascinated with understanding how our brains work. Bob was recently appointed as the Chief Scientist at Cambridge Quantum Computing. Bob thinks that interaction...
#52 - Unadversarial Examples (Hadi Salman, MIT)
May 01, 2021 01:02 - 1 hour - 149 MBPerforming reliably on unseen or shifting data distributions is a difficult challenge for modern vision systems, even slight corruptions or transformations of images are enough to slash the accuracy of state-of-the-art classifiers. When an adversary is allowed to modify an input image directly, models can be manipulated into predicting anything even when there is no perceptible change, this is known an adversarial example. The ideal definition of an adversarial example is when humans consist...
#51 Francois Chollet - Intelligence and Generalisation
April 16, 2021 13:11 - 2 hours - 167 MBIn today's show we are joined by Francois Chollet, I have been inspired by Francois ever since I read his Deep Learning with Python book and started using the Keras library which he invented many, many years ago. Francois has a clarity of thought that I've never seen in any other human being! He has extremely interesting views on intelligence as generalisation, abstraction and an information conversation ratio. He wrote on the measure of intelligence at the end of 2019 and it had a huge impa...
#50 Christian Szegedy - Formal Reasoning, Program Synthesis
April 04, 2021 01:12 - 1 hour - 171 MBDr. Christian Szegedy from Google Research is a deep learning heavyweight. He invented adversarial examples, one of the first object detection algorithms, the inceptionnet architecture, and co-invented batchnorm. He thinks that if you bet on computers and software in 1990 you would have been as right as if you bet on AI now. But he thinks that we have been programming computers the same way since the 1950s and there has been a huge stagnation ever since. Mathematics is the process of taking ...
#49 - Meta-Gradients in RL - Dr. Tom Zahavy (DeepMind)
March 23, 2021 21:36 - 1 hour - 157 MBThe race is on, we are on a collective mission to understand and create artificial general intelligence. Dr. Tom Zahavy, a Research Scientist at DeepMind thinks that reinforcement learning is the most general learning framework that we have today, and in his opinion it could lead to artificial general intelligence. He thinks there are no tasks which could not be solved by simply maximising a reward. Back in 2012 when Tom was an undergraduate, before the deep learning revolution he attended...
#49 - Meta-Gradients in RL - Dr. Tomas Zahavy (DeepMind)
March 23, 2021 21:36 - 1 hour - 157 MBThe race is on, we are on a collective mission to understand and create artificial general intelligence. Dr. Tom Zahavy, a Research Scientist at DeepMind thinks that reinforcement learning is the most general learning framework that we have today, and in his opinion it could lead to artificial general intelligence. He thinks there are no tasks which could not be solved by simply maximising a reward. Back in 2012 when Tom was an undergraduate, before the deep learning revolution he attended...
#48 Machine Learning Security - Andy Smith
March 16, 2021 22:35 - 37 minutes - 51.5 MBFirst episode in a series we are doing on ML DevOps. Starting with the thing which nobody seems to be talking about enough, security! We chat with cyber security expert Andy Smith about threat modelling and trust boundaries for an ML DevOps system. Intro [00:00:00] ML DevOps - a security perspective [00:00:50] Threat Modelling [00:03:03] Adversarial examples? [00:11:27] Nobody understands the whole stack [00:13:53] On the size of the state space, the element of unpredictability [00:18...
047 Interpretable Machine Learning - Christoph Molnar
March 14, 2021 12:34 - 1 hour - 142 MBChristoph Molnar is one of the main people to know in the space of interpretable ML. In 2018 he released the first version of his incredible online book, interpretable machine learning. Interpretability is often a deciding factor when a machine learning (ML) model is used in a product, a decision process, or in research. Interpretability methods can be used to discover knowledge, to debug or justify the model and its predictions, and to control and improve the model, reason about potential b...
#046 The Great ML Stagnation (Mark Saroufim and Dr. Mathew Salvaris)
March 06, 2021 19:47 - 1 hour - 138 MBAcademics think of themselves as trailblazers, explorers — seekers of the truth. Any fundamental discovery involves a significant degree of risk. If an idea is guaranteed to work then it moves from the realm of research to engineering. Unfortunately, this also means that most research careers will invariably be failures at least if failures are measured via “objective” metrics like citations. Today we discuss the recent article from Mark Saroufim called Machine Learning: the great stagnatio...
#045 Microsoft's Platform for Reinforcement Learning (Bonsai)
February 28, 2021 22:59 - 2 hours - 207 MBMicrosoft has an interesting strategy with their new “autonomous systems” technology also known as Project Bonsai. They want to create an interface to abstract away the complexity and esoterica of deep reinforcement learning. They want to fuse together expert knowledge and artificial intelligence all on one platform, so that complex problems can be decomposed into simpler ones. They want to take machine learning Ph.Ds out of the equation and make autonomous systems engineering look more like...
#044 - Data-efficient Image Transformers (Hugo Touvron)
February 25, 2021 22:39 - 52 minutes - 95.9 MBToday we are going to talk about the *Data-efficient image Transformers paper or (DeiT) which Hugo is the primary author of. One of the recipes of success for vision models since the DL revolution began has been the availability of large training sets. CNNs have been optimized for almost a decade now, including through extensive architecture search which is prone to overfitting. Motivated by the success of transformers-based models in Natural Language Processing there has been increasing at...
#043 Prof J. Mark Bishop - Artificial Intelligence Is Stupid and Causal Reasoning won't fix it.
February 19, 2021 11:04 - 1 hour - 131 MBProfessor Mark Bishop does not think that computers can be conscious or have phenomenological states of consciousness unless we are willing to accept panpsychism which is idea that mentality is fundamental and ubiquitous in the natural world, or put simply, that your goldfish and everything else for that matter has a mind. Panpsychism postulates that distinctions between intelligences are largely arbitrary. Mark’s work in the ‘philosophy of AI’ led to an influential critique of computationa...
#042 - Pedro Domingos - Ethics and Cancel Culture
February 11, 2021 01:08 - 1 hour - 129 MBToday we have professor Pedro Domingos and we are going to talk about activism in machine learning, cancel culture, AI ethics and kernels. In Pedro's book the master algorithm, he segmented the AI community into 5 distinct tribes with 5 unique identities (and before you ask, no the irony of an anti-identitarian doing do was not lost on us!). Pedro recently published an article in Quillette called Beating Back Cancel Culture: A Case Study from the Field of Artificial Intelligence. Domingos ha...
#041 - Biologically Plausible Neural Networks - Dr. Simon Stringer
February 03, 2021 20:39 - 1 hour - 119 MBDr. Simon Stringer. Obtained his Ph.D in mathematical state space control theory and has been a Senior Research Fellow at Oxford University for over 27 years. Simon is the director of the the Oxford Centre for Theoretical Neuroscience and Artificial Intelligence, which is based within the Oxford University Department of Experimental Psychology. His department covers vision, spatial processing, motor function, language and consciousness -- in particular -- how the primate visual system learns...
#040 - Adversarial Examples (Dr. Nicholas Carlini, Dr. Wieland Brendel, Florian Tramèr)
January 31, 2021 19:46 - 1 hour - 132 MBAdversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear. there's good reason to believe neural networks look at very different features than we would have expected. As articulated in the 2019 "features not bugs" paper Adversarial examples can be directly attributed to the presence of non-robust features: features derived from patterns in the data distribution that are highly predictive, yet brittle a...
#039 - Lena Voita - NLP
January 23, 2021 23:36 - 1 hour - 108 MBena Voita is a Ph.D. student at the University of Edinburgh and University of Amsterdam. Previously, She was a research scientist at Yandex Research and worked closely with the Yandex Translate team. She still teaches NLP at the Yandex School of Data Analysis. She has created an exciting new NLP course on her website lena-voita.github.io which you folks need to check out! She has one of the most well presented blogs we have ever seen, where she discusses her research in an easily digestable ...
#038 - Professor Kenneth Stanley - Why Greatness Cannot Be Planned
January 20, 2021 01:36 - 2 hours - 152 MBProfessor Kenneth Stanley is currently a research science manager at OpenAI in San Fransisco. We've Been dreaming about getting Kenneth on the show since the very begininning of Machine Learning Street Talk. Some of you might recall that our first ever show was on the enhanced POET paper, of course Kenneth had his hands all over it. He's been cited over 16000 times, his most popular paper with over 3K citations was the NEAT algorithm. His interests are neuroevolution, open-endedness, NNs, ar...
#037 - Tour De Bayesian with Connor Tann
January 11, 2021 01:30 - 1 hour - 87.6 MBConnor Tan is a physicist and senior data scientist working for a multinational energy company where he co-founded and leads a data science team. He holds a first-class degree in experimental and theoretical physics from Cambridge university. With a master's in particle astrophysics. He specializes in the application of machine learning models and Bayesian methods. Today we explore the history, pratical utility, and unique capabilities of Bayesian methods. We also discuss the computational d...
#036 - Max Welling: Quantum, Manifolds & Symmetries in ML
January 03, 2021 18:08 - 1 hour - 94 MBToday we had a fantastic conversation with Professor Max Welling, VP of Technology, Qualcomm Technologies Netherlands B.V. Max is a strong believer in the power of data and computation and its relevance to artificial intelligence. There is a fundamental blank slate paradgm in machine learning, experience and data alone currently rule the roost. Max wants to build a house of domain knowledge on top of that blank slate. Max thinks there are no predictions without assumptions, no generalizati...
#035 Christmas Community Edition!
December 27, 2020 21:59 - 2 hours - 161 MBWelcome to the Christmas special community edition of MLST! We discuss some recent and interesting papers from Pedro Domingos (are NNs kernel machines?), Deepmind (can NNs out-reason symbolic machines?), Anna Rodgers - When BERT Plays The Lottery, All Tickets Are Winning, Prof. Mark Bishop (even causal methods won't deliver understanding), We also cover our favourite bits from the recent Montreal AI event run by Prof. Gary Marcus (including Rich Sutton, Danny Kahneman and Christof Koch). We ...
#034 Eray Özkural- AGI, Simulations & Safety
December 20, 2020 01:16 - 2 hours - 147 MBDr. Eray Ozkural is an AGI researcher from Turkey, he is the founder of Celestial Intellect Cybernetics. Eray is extremely critical of Max Tegmark, Nick Bostrom and MIRI founder Elizier Yodokovsky and their views on AI safety. Eray thinks that these views represent a form of neoludditism and they are capturing valuable research budgets with doomsday fear-mongering and effectively want to prevent AI from being developed by those they don't agree with. Eray is also sceptical of the intelligenc...
#033 Prof. Karl Friston - The Free Energy Principle
December 13, 2020 20:58 - 1 hour - 103 MBThis week Dr. Tim Scarfe, Dr. Keith Duggar and Connor Leahy chat with Prof. Karl Friston. Professor Friston is a British neuroscientist at University College London and an authority on brain imaging. In 2016 he was ranked the most influential neuroscientist on Semantic Scholar. His main contribution to theoretical neurobiology is the variational Free energy principle, also known as active inference in the Bayesian brain. The FEP is a formal statement that the existential imperative for any ...
#032- Simon Kornblith / GoogleAI - SimCLR and Paper Haul!
December 06, 2020 00:43 - 1 hour - 83.3 MBThis week Dr. Tim Scarfe, Sayak Paul and Yannic Kilcher speak with Dr. Simon Kornblith from Google Brain (Ph.D from MIT). Simon is trying to understand how neural nets do what they do. Simon was the second author on the seminal Google AI SimCLR paper. We also cover "Do Wide and Deep Networks learn the same things?", "Whats in a Loss function for Image Classification?", and "Big Self-supervised models are strong semi-supervised learners". Simon used to be a neuroscientist and also gives us t...
#031 WE GOT ACCESS TO GPT-3! (With Gary Marcus, Walid Saba and Connor Leahy)
November 28, 2020 00:40 - 2 hours - 150 MBIn this special edition, Dr. Tim Scarfe, Yannic Kilcher and Keith Duggar speak with Gary Marcus and Connor Leahy about GPT-3. We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations. Note that this podcast version is significantly truncated, watch the youtube version for the TOC and experiments with GPT-3 https://www.youtube.com/watch?v=iccd86vOz3w
#030 Multi-Armed Bandits and Pure-Exploration (Wouter M. Koolen)
November 20, 2020 20:36 - 1 hour - 149 MBThis week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher discuss multi-arm bandits and pure exploration with Dr. Wouter M. Koolen, Senior Researcher, Machine Learning group, Centrum Wiskunde & Informatica. Wouter specialises in machine learning theory, game theory, information theory, statistics and optimisation. Wouter is currently interested in pure exploration in multi-armed bandit models, game tree search, and accelerated learning in sequential decision problems. His research has b...
#029 GPT-3, Prompt Engineering, Trading, AI Alignment, Intelligence
November 08, 2020 18:53 - 1 hour - 102 MBThis week Dr. Tim Scarfe, Dr. Keith Duggar, Yannic Kilcher and Connor Leahy cover a broad range of topics, ranging from academia, GPT-3 and whether prompt engineering could be the next in-demand skill, markets and economics including trading and whether you can predict the stock market, AI alignment, utilitarian philosophy, randomness and intelligence and even whether the universe is infinite! 00:00:00 Show Introduction 00:12:49 Academia and doing a Ph.D 00:15:49 From academia to wall ...
NLP is not NLU and GPT-3 - Walid Saba
November 04, 2020 19:16 - 2 hours - 129 MB#machinelearning This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher speak with veteran NLU expert Dr. Walid Saba. Walid is an old-school AI expert. He is a polymath, a neuroscientist, psychologist, linguist, philosopher, statistician, and logician. He thinks the missing information problem and lack of a typed ontology is the key issue with NLU, not sample efficiency or generalisation. He is a big critic of the deep learning movement and BERTology. We also cover GPT-3 in some d...
AI Alignment & AGI Fire Alarm - Connor Leahy
November 01, 2020 20:31 - 2 hours - 171 MBThis week Dr. Tim Scarfe, Alex Stenlake and Yannic Kilcher speak with AGI and AI alignment specialist Connor Leahy a machine learning engineer from Aleph Alpha and founder of EleutherAI. Connor believes that AI alignment is philosophy with a deadline and that we are on the precipice, the stakes are astronomical. AI is important, and it will go wrong by default. Connor thinks that the singularity or intelligence explosion is near. Connor says that AGI is like climate change but worse, even h...
Kaggle, ML Community / Engineering (Sanyam Bhutani)
October 28, 2020 00:47 - 1 hour - 80.1 MBJoin Dr Tim Scarfe, Sayak Paul, Yannic Kilcher, and Alex Stenlake have a conversation with Mr. Chai Time Data Science; Sanyam Bhutani! 00:00:00 Introduction 00:03:42 Show kick off 00:06:34 How did Sanyam get started into ML 00:07:46 Being a content creator 00:09:01 Can you be self taught without a formal education in ML? 00:22:54 Kaggle 00:33:41 H20 product / job 00:40:58 Intepretability / bias / engineering skills 00:43:22 Get that first job in DS 00:46:29 AWS ML Ops arch...
Sara Hooker - The Hardware Lottery, Sparsity and Fairness
October 20, 2020 22:16 - 1 hour - 83.5 MBDr. Tim Scarfe, Yannic Kilcher and Sayak Paul chat with Sara Hooker from the Google Brain team! We discuss her recent hardware lottery paper, pruning / sparsity, bias mitigation and intepretability. The hardware lottery -- what causes inertia or friction in the marketplace of ideas? Is there a meritocracy of ideas or do the previous decisions we have made enslave us? Sara Hooker calls this a lottery because she feels that machine learning progress is entirely beholdant to the hardware and ...
The Social Dilemma Part 3 - Dr. Rebecca Roache
October 11, 2020 23:16 - 1 hour - 70.8 MBThis week join Dr. Tim Scarfe, Yannic Kilcher, and Keith Duggar have a conversation with Dr. Rebecca Roache in the last of our 3-part series on the social dilemma Netflix film. Rebecca is a senior lecturer in philosophy at Royal Holloway, university of London and has written extensively about the future of friendship. People claim that friendships are not what they used to be. People are always staring at their phones, even when in public Social media has turned us into narcissists who ar...
The Social Dilemma - Part 2
October 06, 2020 19:12 - 1 hour - 98 MBThis week on Machine Learning Street Talk, Dr. Tim Scarfe, Dr. Keith Duggar, Alex Stenlake and Yannic Kilcher have a conversation with the founder and principal researcher at the Montreal AI Ethics institute -- Abhishek Gupta. We cover several topics from the Social Dilemma film and AI Ethics in general. 00:00:00 Introduction 00:03:57 Overcome our weaknesses 00:14:30 threat landscape blind spots 00:18:35 differential reality vs universal shaping 00:24:21 shared reality incentives a...
The Social Dilemma - Part 1
October 03, 2020 21:07 - 1 hour - 62.9 MBIn this first part of our three part series on the Social Dilemma Netflix film, Dr. Tim Scarfe, Yannic "Lightspeed" Kilcher and Zak Jost gang up with Cybersecurity expert Andy Smith. We give you our take on the film. We are super excited to get your feedback on this one! Hope you enjoy. 00:00:00 Introduction 00:06:11 Moral hypocrisy 00:12:38 Road to hell is paved with good intentions, attention economy 00:15:04 They know everything about you 00:18:02 Addiction 00:21:22 Differenti...
Capsule Networks and Education Targets
September 29, 2020 19:13 - 1 hour - 77.9 MBIn today's episode, Dr. Keith Duggar, Alex Stenlake and Dr. Tim Scarfe chat about the education chapter in Kenneth Stanley's "Greatness cannot be planned" book, and we relate it to our Algoshambes conversation a few weeks ago. We debate whether objectives in education are a good thing and whether they cause perverse incentives and stifle creativity and innovation. Next up we dissect capsule networks from the top down! We finish off talking about fast algorithms and quantum computing. 00:00:...
Programming Languages, Software Engineering and Machine Learning
September 25, 2020 17:32 - 1 hour - 77.7 MBThis week Dr. Tim Scarfe, Dr. Keith Duggar, Yannic "Lightspeed" Kilcher have a conversation with Microsoft Senior Software Engineer Sachin Kundu. We speak about programming languages including which our favourites are and functional programming vs OOP. Next we speak about software engineering and the intersection of software engineering and machine learning. We also talk about applications of ML and finally what makes an exceptional software engineer and tech lead. Sachin is an expert in thi...
Computation, Bayesian Model Selection, Interactive Articles
September 22, 2020 22:03 - 1 hour - 68.2 MBThis week Dr. Keith Duggar, Alex Stenlake and Dr. Tim Scarfe discuss the theory of computation, intelligence, Bayesian model selection, the intelligence explosion and the the phenomenon of "interactive articles". 00:00:00 Intro 00:01:27 Kernels and context-free grammars 00:06:04 Theory of computation 00:18:41 Intelligence 00:22:03 Bayesian model selection 00:44:05 AI-IQ Measure / Intelligence explosion 00:52:09 Interactive articles 01:12:32 Outro
Kernels!
September 18, 2020 17:54 - 1 hour - 90.1 MBToday Yannic Lightspeed Kilcher and I spoke with Alex Stenlake about Kernel Methods. What is a kernel? Do you remember those weird kernel things which everyone obsessed about before deep learning? What about Representer theorem and reproducible kernel hilbert spaces? SVMs and kernel ridge regression? Remember them?! Hope you enjoy the conversation! 00:00:00 Tim Intro 00:01:35 Yannic clever insight from this discussion 00:03:25 Street talk and Alex intro 00:05:06 How kernels are taught ...
Explainability, Reasoning, Priors and GPT-3
September 16, 2020 13:34 - 1 hour - 79.6 MBThis week Dr. Tim Scarfe and Dr. Keith Duggar discuss Explainability, Reasoning, Priors and GPT-3. We check out Christoph Molnar's book on intepretability, talk about priors vs experience in NNs, whether NNs are reasoning and also cover articles by Gary Marcus and Walid Saba critiquing deep learning. We finish with a brief discussion of Chollet's ARC challenge and intelligence paper. 00:00:00 Intro 00:01:17 Explainability and Christoph Molnars book on Intepretability 00:26:45 Explainabil...
SWaV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments (Mathilde Caron)
September 14, 2020 00:22 - 1 hour - 80.3 MBThis week Dr. Tim Scarfe, Yannic Lightspeed Kicher, Sayak Paul and Ayush Takur interview Mathilde Caron from Facebook Research (FAIR). We discuss Mathilde's paper which she wrote with her collaborators "SWaV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments" @ https://arxiv.org/pdf/2006.09882.pdf This paper is the latest unsupervised contrastive visual representations algorithm and has a new data augmentation strategy and also a new online clustering strategy. ...
UK Algoshambles, Neuralink, GPT-3 and Intelligence
September 07, 2020 09:40 - 1 hour - 88 MBThis week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic "Lightspeed" Kilcher respond to the "Algoshambles" exam fiasco in the UK where the government were forced to step in to standardise the grades which were grossly inflated by the schools. The schools and teachers are all paid on metrics related to the grades received by students, what could possibly go wrong?! The result is that we end up with grades which have lost all their value and students are coached for the exams and don't actually...
Sayak Paul
July 17, 2020 10:04 - 1 hour - 88.1 MBThis week we spoke with Sayak Paul, who is extremely active in the machine learning community. We discussed the AI landscape in India, unsupervised representation learning, data augmentation and contrastive learning, explainability, abstract scene representations and finally pruning and the recent super positions paper. I really enjoyed this conversation and I hope you folks do too! 00:00:00 Intro to Sayak 00:17:50 AI landscape in India 00:24:20 Unsupervised representation learning 00:26...
Robert Lange on NN Pruning and Collective Intelligence
July 08, 2020 12:27 - 1 hour - 98.1 MBWe speak with Robert Lange! Robert is a PhD student at the Technical University Berlin. His research combines Deep Multi-Agent Reinforcement Learning and Cognitive Science to study the learning dynamics of large collectives. He has a brilliant blog where he distils and explains cutting edge ML research. We spoke about his story, economics, multi-agent RL, intelligence and AGI, and his recent article summarising the state of the art in neural network pruning. Robert's article on pruning in...
WelcomeAIOverlords (Zak Jost)
June 30, 2020 12:39 - 1 hour - 109 MBWe welcome Zak Jost from the WelcomeAIOverlords channel. Zak is an ML research scientist at Amazon. He has a great blog at http://blog.zakjost.com and also a Discord channel at https://discord.gg/xh2chKX WelcomeAIOverlords: https://www.youtube.com/channel/UCxw9_WYmLqlj5PyXu2AWU_g 00:00:00 INTRO START 00:01:07 MAIN SHOW START 00:01:59 ZAK'S STORY 00:05:06 YOUTUBE DISCUSSION 00:24:12 UNDERSTANDING PAPERS 00:29:53 CONTRASTIVE LEARNING INTRO 00:33:00 BRING YOUR OWN LATENT PAPER 01:03:1...
Facebook Research - Unsupervised Translation of Programming Languages
June 24, 2020 16:50 - 1 hour - 57.7 MBIn this episode of Machine Learning Street Talk Dr. Tim Scarfe, Yannic Kilcher and Connor Shorten spoke with Marie-Anne Lachaux, Baptiste Roziere and Dr. Guillaume Lample from Facebook Research (FAIR) in Paris. They recently released the paper "Unsupervised Translation of Programming Languages" which was an exciting new approach to learned translation of programming languages (learned transcoder) using an unsupervised encoder trained on individual monolingual corpora i.e. no parallel languag...
Francois Chollet - On the Measure of Intelligence
June 19, 2020 00:35 - 2 hours - 141 MBWe cover Francois Chollet's recent paper. Abstract; To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans. Over the past hundred years, there has been an abundance of attempts to define and measure intelligence, across both the fields of psychology and AI...
OpenAI GPT-3: Language Models are Few-Shot Learners
June 06, 2020 23:42 - 1 hour - 154 MBIn this episode of Machine Learning Street Talk, Tim Scarfe, Yannic Kilcher and Connor Shorten discuss their takeaways from OpenAI’s GPT-3 language model. With the help of Microsoft’s ZeRO-2 / DeepSpeed optimiser, OpenAI trained an 175 BILLION parameter autoregressive language model. The paper demonstrates how self-supervised language modelling at this scale can perform many downstream tasks without fine-tuning. 00:00:00 Intro 00:00:54 ZeRO1+2 (model + Data parallelism) (Connor) 00:03:17 ...
Jordan Edwards: ML Engineering and DevOps on AzureML
June 03, 2020 00:31 - 1 hour - 100 MBThis week we had a super insightful conversation with Jordan Edwards, Principal Program Manager for the AzureML team! Jordan is on the coalface of turning machine learning software engineering into a reality for some of Microsoft's largest customers. ML DevOps is all about increasing the velocity of- and orchastrating the non-interactive phase of- software deployments for ML. We cover ML DevOps and Microsoft Azure ML. We discuss model governance, testing, intepretability, tooling. We cov...
One Shot and Metric Learning - Quadruplet Loss (Machine Learning Dojo)
June 02, 2020 11:30 - 2 hours - 136 MB*Note this is an episode from Tim's Machine Learning Dojo YouTube channel. Join Eric Craeymeersch on a wonderful discussion all about ML engineering, computer vision, siamese networks, contrastive loss, one shot learning and metric learning. 00:00:00 Introduction 00:11:47 ML Engineering Discussion 00:35:59 Intro to the main topic 00:42:13 Siamese Networks 00:48:36 Mining strategies 00:51:15 Contrastive Loss 00:57:44 Trip loss paper 01:09:35 Quad loss paper 01:25:49 Eric's Quadlo...
Harri Valpola: System 2 AI and Planning in Model-Based Reinforcement Learning
May 25, 2020 11:00 - 1 hour - 136 MBIn this episode of Machine Learning Street Talk, Tim Scarfe, Yannic Kilcher and Connor Shorten interviewed Harri Valpola, CEO and Founder of Curious AI. We continued our discussion of System 1 and System 2 thinking in Deep Learning, as well as miscellaneous topics around Model-based Reinforcement Learning. Dr. Valpola describes some of the challenges of modelling industrial control processes such as water sewage filters and paper mills with the use of model-based RL. Dr. Valpola and his coll...