Energy-Based Dropout in Restricted Boltzmann Machines: Why Not Go Random

Página do item simplificado
dc.contributor.authorRoder, Mateus
dc.contributor.authorde Rosa, Gustavo Henrique
dc.contributor.authorde Albuquerque, Victor Hugo C.
dc.contributor.authorRossi, Andre L. D.
dc.contributor.authorPapa, Joao P.
dc.contributor.institutionUniversidade Estadual Paulista (Unesp)
dc.date.accessioned2021-06-25T10:19:27Z
dc.date.available2021-06-25T10:19:27Z
dc.date.issued2020-01-01
dc.description.abstractDeep learning architectures have been widely fostered throughout the last years, being used in a wide range of applications, such as object recognition, image reconstruction, and signal processing. Nevertheless, such models suffer from a common problem known as overfitting, which limits the network from predicting unseen data effectively. Regularization approaches arise in an attempt to address such a shortcoming. Among them, one can refer to the well-known Dropout, which tackles the problem by randomly shutting down a set of neurons and their connections according to a certain probability. Therefore, this approach does not consider any additional knowledge to decide which units should be disconnected. In this paper, we propose an energy-based Dropout (E-Dropout) that makes conscious decisions whether a neuron should be dropped or not. Specifically, we design this regularization method by correlating neurons and the model’s energy as an importance level for further applying it to energy-based models, such as Restricted Boltzmann Machines (RBMs). The experimental results over several benchmark datasets revealed the proposed approach’s suitability compared to the traditional Dropout and the standard RBMs.en
dc.description.affiliationSão Paulo State University, Sao Paulo, SP 17033360 Brazil (e-mail: mateus.roder@unesp.br).
dc.description.affiliationSão Paulo State University, Sao Paulo, SP 17033360 Brazil (e-mail: gustavo.rosa@unesp.br).
dc.description.affiliationARMTEC Tecnologia em Robótica, Fortaleza, /CE 60150000 Brazil (e-mail: victor.albuquerque@ieee.org).
dc.description.affiliationSão Paulo State University, Sao Paulo, SP 17033360 Brazil (e-mail: andre.rossi@unesp.br).
dc.description.affiliationSão Paulo State University, Sao Paulo, SP 17033360 Brazil (e-mail: joao.papa@unesp.br).
dc.identifierhttp://dx.doi.org/10.1109/TETCI.2020.3043764
dc.identifier.citationIEEE Transactions on Emerging Topics in Computational Intelligence.
dc.identifier.doi10.1109/TETCI.2020.3043764
dc.identifier.issn2471-285X
dc.identifier.scopus2-s2.0-85098746260
dc.identifier.urihttp://hdl.handle.net/11449/205676
dc.language.isoeng
dc.relation.ispartofIEEE Transactions on Emerging Topics in Computational Intelligence
dc.sourceScopus
dc.subjectComputational modeling
dc.subjectDropout
dc.subjectenergy-based dropout
dc.subjectImage reconstruction
dc.subjectmachine learning
dc.subjectMathematical model
dc.subjectNeurons
dc.subjectregularization
dc.subjectrestricted boltzmann machines
dc.subjectStandards
dc.subjectTask analysis
dc.subjectTraining
dc.titleEnergy-Based Dropout in Restricted Boltzmann Machines: Why Not Go Randomen
dc.typeArtigo

Arquivos

Coleções

Artigos