Generalized Language Models
This article discusses the evolution of generalized language models including BERT, GPT, and other major pre-trained models that achieved state-of-the-art results on various NLP tasks. The piece covers the breakthrough progress in 2018 with large-scale unsupervised pre-training approaches that don't require labeled data, similar to how ImageNet helped computer vision.