Distributed deep networks based on Bagging-Down SGD algorithm

© 2019, Editorial Office of Systems Engineering and Electronics. All right reserved. As a cutting-edge disruptive technology, deep learning and unsupervised learning have attracted a significant research attention, and it has been widely acknowledged that training big data with a distributed deep learning algorithm can get better structures. However, there are two main problems with traditional distributed deep learning algorithms: the speed of training is slow and the accuracy of training is low. The Bootstrap aggregating-down stochastic gradient descent (Bagging-Down SGD) algorithm is proposed to solve the speed problem mainly. We add a speed controller to update the parameters of the single machine statistically, and to split model training and parameters updating to improve the training speed with the assurance of the same accuracy. It is to be proved in the experiment that the algorithm has the generality to learn the structures of different kinds of data. This article is in Chinese.