Towards a conversational agent that can chat about...anything

Abstract

Motivated by the need for chatbots to engage in conversations on a wide range of topics, researchers have created Meena, a conversational agent trained on 2.6 billion parameters and 341 GB of text data. Meena utilizes Evolved Transformer architecture and minimizes perplexity as its training objective. To assess conversational quality, a novel metric called Sensibleness and Specificity Average (SSA) has been developed, comprising judgments on specific and sensible responses. Meena outperforms existing chatbots in terms of SSA, exhibiting a strong correlation between low perplexity and high SSA scores. This correlation opens up the possibility of reliable evaluation via perplexity, facilitating faster model development. – AI-generated abstract.