Variational studies and replica symmetry breaking in the generalization problem of the binary perceptron

Página do item simplificado
dc.contributor.authorBotelho, Evaldo
dc.contributor.authorMattos, Cristiano R. [UNESP]
dc.contributor.authorCaticha, Nestor
dc.contributor.institutionUniversidade de São Paulo (USP)
dc.contributor.institutionUniversidade Estadual Paulista (Unesp)
dc.date.accessioned2014-05-27T11:19:57Z
dc.date.available2014-05-27T11:19:57Z
dc.date.issued2000-11-01
dc.description.abstractWe analyze the average performance of a general class of learning algorithms for the nondeterministic polynomial time complete problem of rule extraction by a binary perceptron. The examples are generated by a rule implemented by a teacher network of similar architecture. A variational approach is used in trying to identify the potential energy that leads to the largest generalization in the thermodynamic limit. We restrict our search to algorithms that always satisfy the binary constraints. A replica symmetric ansatz leads to a learning algorithm which presents a phase transition in violation of an information theoretical bound. Stability analysis shows that this is due to a failure of the replica symmetric ansatz and the first step of replica symmetry breaking (RSB) is studied. The variational method does not determine a unique potential but it allows construction of a class with a unique minimum within each first order valley. Members of this class improve on the performance of Gibbs algorithm but fail to reach the Bayesian limit in the low generalization phase. They even fail to reach the performance of the best binary, an optimal clipping of the barycenter of version space. We find a trade-off between a good low performance and early onset of perfect generalization. Although the RSB may be locally stable we discuss the possibility that it fails to be the correct saddle point globally. ©2000 The American Physical Society.en
dc.description.affiliationInstituto de Física Universidade de São Paulo, Caixa Postal 66318, São Paulo, SP 05315-970
dc.description.affiliationFaculdade de Engenharia Universidade Estadual Paulista, Caixa Postal 205, Guaratinguetá, SP 12500-000
dc.description.affiliationUnespFaculdade de Engenharia Universidade Estadual Paulista, Caixa Postal 205, Guaratinguetá, SP 12500-000
dc.format.extent6999-7007
dc.identifierhttp://dx.doi.org/10.1103/PhysRevE.62.6999
dc.identifier.citationPhysical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics, v. 62, n. 5 B, p. 6999-7007, 2000.
dc.identifier.doi10.1103/PhysRevE.62.6999
dc.identifier.file2-s2.0-0034318079.pdf
dc.identifier.issn1063-651X
dc.identifier.scopus2-s2.0-0034318079
dc.identifier.urihttp://hdl.handle.net/11449/66275
dc.identifier.wosWOS:000165341900038
dc.language.isoeng
dc.relation.ispartofPhysical Review E: Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics
dc.rights.accessRightsAcesso restrito
dc.sourceScopus
dc.subjectComputer simulation
dc.subjectEntropy
dc.subjectFailure analysis
dc.subjectGibbs free energy
dc.subjectIntegration
dc.subjectLearning algorithms
dc.subjectMonte Carlo methods
dc.subjectNeural networks
dc.subjectPhase transitions
dc.subjectPolynomials
dc.subjectPotential energy
dc.subjectThermodynamic stability
dc.subjectBinary perceptrons
dc.subjectReplica symmetry breaking (RSB)
dc.subjectStatistical mechanics
dc.titleVariational studies and replica symmetry breaking in the generalization problem of the binary perceptronen
dc.typeArtigo
dcterms.licensehttp://publish.aps.org/authors/transfer-of-copyright-agreement

Arquivos

Pacote Original
Agora exibindo 1 - 1 de 1
Imagem de Miniatura
Nome:
2-s2.0-0034318079.pdf
Tamanho:
189.23 KB
Formato:
Adobe Portable Document Format
Baixar

Coleções

Artigos