A New Concept of Convex based Multiple Neural Networks Structure

In this paper, a new concept of convex based multiple neural networks structure is proposed. This new approach uses the collective information from multiple neural networks to train the model. From both theoretical and experimental analysis, it is going to demonstrate that the new approach gives a faster training speed of convergence with a similar or even better test accuracy, compared to a conventional neural network structure. Two experiments are conducted to demonstrate the performance of our new structure: the first one is a semantic frame parsing task for spoken language understanding (SLU) on ATIS dataset, and the other is a hand written digits recognition task on MNIST dataset. We test this new structure using both recurrent neural network and convolutional neural networks through these two tasks. The results of both experiments demonstrate a 4x-8x faster training speed with better or similar performance by using this new concept.