Entropy-Based Filter Selection in CNNs Applied to Text Classification

Filter selection in convolutional neural networks aims at finding the most important filters in a convolutional layer, with the goal of reducing computational costs and needed storage, as well as understanding the networks’ inner workings. In this paper we propose an entropy-based filter selection method that ranks filters based on the mutual information between their activations and the output classes using validation data. Our proposed method outperforms using filters’ absolute weights sum by a large margin, allowing to regain better performance with fewer filters.

Keywords

Convolutional neural networks Filter pruning Mutual information

Language

English

Citation

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), v. 12319 LNAI, p. 497-510.