56 - Deep contextualized word representations, with Matthew Peters

NLP Highlights

English - April 04, 2018 21:22 - 30 minutes - 27.5 MB - ★★★★★ - 22 ratings
Science Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed

Previous Episode: 55 - Matchbox: Dispatch-driven autobatching for imperative deep learning, with James Bradbury

Next Episode: 57 - A Survey Of Cross-lingual Word Embedding Models, with Sebastian Ruder

NAACL 2018 paper, by Matt Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Chris Clark, Kenton Lee, and Luke Zettlemoyer.

In this episode, AI2's own Matt Peters comes on the show to talk about his recent work on ELMo embeddings, what some have called "the next word2vec". Matt has shown very convincingly that using a pre-trained bidirectional language model to get contextualized word representations performs substantially better than using static word vectors. He comes on the show to give us some more intuition about how and why this works, and to talk about some of the other things he tried and what's coming next.

https://www.semanticscholar.org/paper/Deep-contextualized-word-representations-Peters-Neumann/4b17597b856c087f109381ce77d60d9017cb6f9a