Fine-Tuning Temperatures in Restricted Boltzmann Machines Using Meta-Heuristic Optimization
Nenhuma Miniatura disponível
Data
2020-07-01
Autores
Roder, Mateus [UNESP]
De Rosa, Gustavo Henrique [UNESP]
Papa, Joao Paulo [UNESP]
Breve, Fabricio Aparecido [UNESP]
Título da Revista
ISSN da Revista
Título de Volume
Editor
Resumo
Restricted Boltzmann Machines (RBM) are stochastic neural networks mainly used for image reconstruction and unsupervised feature learning. An enhanced version, the temperature-based RBM (T-RBM), considers a new temperature parameter during the learning process that influences the neurons' activation. Nevertheless, the major vulnerability of such models concerns selecting an adequate system's temperature, which might lead them to inadequate training or even overfitting when wrongly set, thus limiting the network from predicting or working effectively over unseen data. This paper addresses the problem of selecting a suitable system's temperature through a meta-heuristic optimization process. Meta-heuristic-driven techniques, such as Particle Swarm Optimization, Bat Algorithm, and Artificial Bee Colony are employed to find proper values for the temperature parameter. Additionally, for comparison purposes, three standard temperature values and a random search are used as baselines. The results revealed that optimizing T-RBM is suitable for training purposes, primarily due to their complex fitness landscape, which makes fine-tuning temperatures a nontrivial task.
Descrição
Palavras-chave
Image Reconstruction, Meta-Heuristic Optimization, Restricted Boltzmann Machine, Temperature-based Systems
Como citar
2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings.