A step toward improving the performance of deep learning by adding rules to node propagation, this paper is an interesting contribution to the area of neural networks.

The purpose of the proposed research is to reduce the computational complexity and the memory accesses of deep learning algorithms by network pruning. The authors go further in their account and propose to structure the process of pruning, for example, to introduce rules guiding and deciding which branch of the network to be pruned. The introduced structured sparsity at various scales for convolutional neural networks, for example mapwise, kernelwise, and intrakernel strided sparsity, bring savings of computational resources. Further, the method filters the network connections by assessing the importance weight of each particle based on misclassification rate according to a corresponding connectivity pattern. The paper shows that the intrakernel strided sparsity with a simple constraint can significantly reduce the size of the kernel and the feature map tensors.

A very well-described method with extensive evaluation and explanation of evaluation results, and eloquent convincing arguments about the advantages of the approach, for example the ability to prune 70 percent of the network with 1 percent loss of accuracy, this paper is good reading for students, scholars, and practitioners interested in deep learning, neural network optimization, and real-world applications using deep learning.