Análise de aprendizado adversarial baseado em similaridade na geração de texto
Carregando...
Arquivos
Data
2022-11-22
Autores
Rosa, Gustavo Henrique de
Título da Revista
ISSN da Revista
Título de Volume
Editor
Universidade Estadual Paulista (Unesp)
Resumo
Algoritmos de Aprendizado de Máquina têm sido amplamente fomentados nos últimos anos, principalmente devido às suas capacidades discriminativas em problemas de Visão Computacional e Processamento de Linguagem Natural. Ademais, suas capacidades generativas permitiram aplicações em tarefas de natureza discreta (sequências de caracteres e palavras), isto é, geração de texto. Uma arquitetura em específico, denotada por Redes Adversariais Generativas, utiliza uma estrutura composta por discriminador e gerador, os quais procuram obter um equilíbrio entre gerar dados artificiais e classificá-los como dados verdadeiros. Recentemente, vários trabalhos propuseram soluções baseadas em Redes Adversariais Generativas para a geração de texto, porém apenas alguns deles conseguiram gerar textos sem palavras repetidas e com algum significado semântico. Um problema decorrente das Redes Adversariais Generativas consiste na dificuldade em estabelecer um equilíbrio no treinamento e, consequentemente, gerar textos artificiais que assemelham-se aos textos reais. Desta forma, a presente tese aprimora o desenvolvimento de modelos adversariais textuais através de funções de similaridade aprendidas por Redes Siamesas, as quais fornecem recompensas capazes de melhor distinguir entre textos artificiais e reais. Adicionalmente, tais modelos são aperfeiçoados com o uso da otimização meta-heurística, a qual fornece conjuntos de hiperparâmetros específicos para as tarefas em questão. Os resultados experimentais obtidos demonstram a capacidade da arquitetura proposta, denotada por Rede Adversarial Generativa por Similaridade Textual, do inglês Text-Similarity Generative Adversarial Network (TS-GAN), em quatro base de dados da literatura. As TS-GANs atingiram resultados superiores às arquiteturas adversariais estado-da-arte e, em suas versões pós-otimização, foram capazes de aprimorar as métricas de suas versões iniciais (sem otimização) em duas de quatro bases de dados.
Machine Learning algorithms have been paramount in the latest years, mainly due to their discriminative capacity in Computer Vision and Natural Language Processing tasks. Furthermore, their generative potentials allowed usage in discrete-based (sequences of characters and words) tasks, such as text generation. A specific architecture denoted as Generative Adversarial Networks uses an structure composed of discriminators and generators to establish an equilibrium between artificial data generation and their classification as real data. Several works proposed adversarial-based models to generate text; however, only a few could generate non-repeated text with little semantic significance. Furthermore, a recurring issue regarding Generative Adversarial Networks consists of the difficulty of establishing a training equilibrium and, consequently, generating artificial text that resembles the original ones. Therefore, this thesis enhances the development of text-based adversarial models through similarity functions learned from Siamese Networks, which provides rewards capable of better distinguishing between artificial and original texts. Such models are improved through meta-heuristic optimization, which furnishes specific hyperparameters to the accounted tasks. The experimental results indicate the capacity of the proposed architecture, denoted by Text-Similarity Generative Adversarial Network (TS-GAN), amongst four literature datasets. The TS-GANs obtained state-of-the-art results and, in their post-optimization versions, were able to improve their standard versions (without optimization) in two out of four datasets.
Machine Learning algorithms have been paramount in the latest years, mainly due to their discriminative capacity in Computer Vision and Natural Language Processing tasks. Furthermore, their generative potentials allowed usage in discrete-based (sequences of characters and words) tasks, such as text generation. A specific architecture denoted as Generative Adversarial Networks uses an structure composed of discriminators and generators to establish an equilibrium between artificial data generation and their classification as real data. Several works proposed adversarial-based models to generate text; however, only a few could generate non-repeated text with little semantic significance. Furthermore, a recurring issue regarding Generative Adversarial Networks consists of the difficulty of establishing a training equilibrium and, consequently, generating artificial text that resembles the original ones. Therefore, this thesis enhances the development of text-based adversarial models through similarity functions learned from Siamese Networks, which provides rewards capable of better distinguishing between artificial and original texts. Such models are improved through meta-heuristic optimization, which furnishes specific hyperparameters to the accounted tasks. The experimental results indicate the capacity of the proposed architecture, denoted by Text-Similarity Generative Adversarial Network (TS-GAN), amongst four literature datasets. The TS-GANs obtained state-of-the-art results and, in their post-optimization versions, were able to improve their standard versions (without optimization) in two out of four datasets.
Descrição
Palavras-chave
Aprendizado de máquina, Aprendizado adversarial, Processamento de linguagem natural, Geração de texto, Redes siamesas, Otimização meta-heurística, Machine learning, Adversarial learning, Natural language processing, Text generation, Siamese networks, Meta-heuristic optimization