NADiA - Towards Neural Network Driven Virtual Human Conversation Agents

Advances in artificial intelligence and machine learning-in particular neural networks-have given rise to a new generation of virtual assistants and chatbots. Within this work, we describe the motivation and architecture of NADiA-Neurally Animated Dialog Agentwhich leverages both the user's verbal input and facial expressions for multi-modal conversation. NADiA combines a neural language model that generates conversational responses, a convolutional neural network for facial expression analysis, and virtual human technology that is deployed on a mobile phone.