![NLP Highlights artwork](https://is1-ssl.mzstatic.com/image/thumb/Podcasts127/v4/a8/49/90/a849903a-65af-d8fc-07a7-c0d1bbf826a6/mza_4767231250788281707.jpg/100x100bb.jpg)
114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro
NLP Highlights
English - May 26, 2020 22:15 - 43 minutes - 39.8 MB - ★★★★★ - 22 ratingsScience Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed
Previous Episode: 113 - Managing Industry Research Teams, with Fernando Pereira
Next Episode: 115 - AllenNLP, interviewing Matt Gardner
We invited Marco Tulio Ribeiro, a Senior Researcher at Microsoft, to talk about evaluating NLP models using behavioral testing, a framework borrowed from Software Engineering. Marco describes three kinds of black-box tests the check whether NLP models satisfy certain necessary conditions. While breaking the standard IID assumption, this framework presents a way to evaluate whether NLP systems are ready for real-world use. We also discuss what capabilities can be tested using this framework, how one can come up with good tests, and the need for an evolving set of behavioral tests for NLP systems.
Marco’s homepage: https://homes.cs.washington.edu/~marcotcr/