Fine-tuning restricted Boltzmann machines using quaternions and its application for spam detection

Abstract

Restricted Boltzmann Machines (RBMs) have been used in a number of applications, but only a few works have addressed them in the context of information security. However, such models have their performance severely affected by some hyperparameters that are usually hand-tuned. In this work, the authors consider learning features in an unsupervised fashion by means of RBMs fine-tuned by hypercomplex-based metaheuristic techniques in the context of malicious content detection. Experiments are conducted over three public datasets and six metaheuristic techniques, which are used to fine-tune RBM hyperparameters such that RBM extracts features that best represent malicious content present in spam e-mail messages, and generates a dataset to be used as input to classification through the Optimum Path Forest supervised algorithm. Experimental results demonstrate that a small number of features generated through RBM can achieve a competitive accuracy in relation to the original dataset, however, with lower computational cost. Furthermore, this study presents the power of quaternions for RBMs parameter optimisation, comparing it against the well-known Harmonic Search, as well as its variants Improved Harmonic Search and Parameter Setting-Free Harmonic Search. It was concluded that RBM-based learning techniques are suitable for features extraction in the context of this work.