BERT: one NLP model to rule them all (Practical AI #22)

Fully Connected – a series where Chris and Daniel keep you up to date with everything that’s happening in the AI community. This week we discuss BERT, a new method of pre-training language representations from Google for natural language processing (NLP) tasks. Then we tackle Facebook’s Horizon, the first open source reinforcement learning platform for large-scale products and services. We also address synthetic data, and suggest a few learning resources.

Fully Connected – a series where Chris and Daniel keep you up to date with everything that’s happening in the AI community.

This week we discuss BERT, a new method of pre-training language representations from Google for natural language processing (NLP) tasks. Then we tackle Facebook’s Horizon, the first open source reinforcement learning platform for large-scale products and services. We also address synthetic data, and suggest a few learning resources.

Leave us a comment

Changelog++ members support our work, get closer to the metal, and make the ads disappear. Join today!

Sponsors:

Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
Rollbar – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog.
Linode – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog
Algolia – Our search partner. Algolia’s full suite search APIs enable teams to develop unique search and discovery experiences across all platforms and devices. We’re using Algolia to power our site search here at Changelog.com. Get started for free and learn more at algolia.com.

Featuring:

Chris Benson – Twitter, GitHub, LinkedIn, WebsiteDaniel Whitenack – Twitter, GitHub, Website

Show Notes:

News/Discussion:

Is artificial intelligence set to become art’s next medium?
BERT (a new method for obtaining rich contextual language representations during pre-training):

Google research article
TensorFlow BERT
BERT paper
PyTorch BERT
NY Times article
Example Colab notebook
BERT explained article
Transformer paper
Google Open Sources BERT to Train Natural Language Models Without Breaking the Bank

Horizon: The first open source reinforcement learning platform for large-scale products and services
Does Synthetic Data Hold The Secret To Artificial Intelligence?
AI Experts: Moving forward with AI likely a series of small steps, not giant leaps

Learning resources:

The Backpropagation Algorithm Demystified

Books

“Grokking Deep Learning” by Andrew Trask

Something missing or broken? PRs welcome!