When was the last time you had a great conversation? A good conversation requires various skills: engaging talking points, exhibiting empathy etc. This article explores the the development of large scale models that can learn these skills for a open-domain chat. For more details, read the original paper.

This blogpost explores the relationship between predictive uncertainity and language generation, and the correlation between uncertainity and hallucination in NLG. For more details, read the original paper.

Existing language representation models like word2vec, BERT, XLNet etc., are based on the co-occurrence of tokens or sequences. Language, however, is more complex than that. It contains features that are not captured by just co-occurence counts! ERNIE 2.0 provides a framework to learn to better represent language by using a suite of lexical, syntactic, and semantic pre-training tasks.

Written with Josh Doidge.