Reconhecimento de ações humanas baseado em articulações do esqueleto obtidas de poses 2D
Carregando...
Arquivos
Data
2023-02-03
Autores
Belluzzo, Bruno
Título da Revista
ISSN da Revista
Título de Volume
Editor
Universidade Estadual Paulista (Unesp)
Resumo
Com o aumento da capacidade das tecnologias atuais de armazenamento e processamento de grandes volumes de dados em uma velocidade cada vez maior, a análise e o reconhecimento de padrões em vídeos passaram a ser pesquisadas e empregadas nas mais diversas aplicações, dentre as quais o reconhecimento automático de ações humanas, que visa identificar em um determinado vídeo as ações sendo executadas pelas pessoas presentes, seja para fins recreativos ou para o monitoramento e a segurança em locais públicos ou até mesmo privados. Detectar pessoas nos vídeos e reconhecer as ações sendo realizadas por elas é uma tarefa complexa, pois exige a extração de caracteríticas que representam um padrão de movimentos realizados pela pessoa tanto no aspecto espacial, quanto no aspecto temporal, ao longo dos diversos \textit{frames} do vídeo. Uma maneira de obter informações que descrevam o movimento do corpo humano em vídeos é identificar as articulações do esqueleto humano nos diversos \textit{frames}, o que pode ser realizado utilizando-se algoritmos de estimação de pose 2D em imagens. Atualmente, existem algoritmos bastante eficazes e eficientes disponíveis, capazes de detectar as articulações do corpo humano e retornarem suas coordenadas nas imagens. Aliado a isso, tem se observado nos últimos anos uma grande evolução dos métodos e algoritmos de aprendizado de máquina, destinados ao reconhecimento de padrões complexos, inspirados em modelos biológicos, com ênfase nos métodos baseados em aprendizado de máquina profundo e recorrente. Esta dissertação de mestrado tem como objetivo propor um método de reconhecimento de ações humanas em vídeo baseado nas articulações dos esqueletos obtidas de poses 2D estimadas por meio de algoritmos estado da arte, utilizando redes neurais recorrentes convolucionais para propiciar mais robustez ao processo. O método proposto foi avaliado utilizando-se duas bases de dados públicas e populares de vídeos de ações humanas, a KTH e a Weizmann. Os resultados obtidos foram superiores aos resultados obtidos por vários métodos encontrados na literatura e comparáveis à métodos estado-da-arte, com a vantagem de apresentar uma estratégia simples para a extração de características a partir das articulações dos esqueletos obtidas das poses 2D.
With the increase in the capacity of current technologies for storing and processing large volumes of data at an ever-increasing speed, the analysis and recognition of patterns in videos began to be researched and used in the most diverse applications, among which automatic recognition of human actions, which aims to identify in a given video the actions being performed by the people present, whether for recreational purposes or for monitoring and security in public or even private places. Detecting people in the videos and recognizing their actions is a complex task, as it requires the extraction of features that represent a pattern of movements performed by the person both in the spatial and temporal aspects, along the different frames of the video. One way to obtain information describing the movement of the human body in videos is to identify the joints of the human skeleton in the different frames, which can be done using 2D pose estimation algorithms in images. Currently, there are very effective and efficient algorithms available, capable of detecting the joints of the human body and returning their coordinates in the images. Allied with this, there has been a great evolution in machine learning methods and algorithms in recent years, aimed at recognizing complex patterns, inspired by biological models, with emphasis on methods based on deep and recurrent machine learning. This master's thesis aimed to propose a method for recognizing human actions in video based on skeletal joints obtained from 2D poses estimated using state-of-the-art algorithms, using deep machine learning methods and recurrent neural networks to provide more robustness to the process. The proposed method was evaluated using two public and popular databases of videos of human actions, KTH and Weizmann. The results obtained were superior to several methods found in the literature and comparable to state-of-the-art methods, with the advantage of presenting a simple strategy for extracting features from skeletal joints obtained from 2D poses.
With the increase in the capacity of current technologies for storing and processing large volumes of data at an ever-increasing speed, the analysis and recognition of patterns in videos began to be researched and used in the most diverse applications, among which automatic recognition of human actions, which aims to identify in a given video the actions being performed by the people present, whether for recreational purposes or for monitoring and security in public or even private places. Detecting people in the videos and recognizing their actions is a complex task, as it requires the extraction of features that represent a pattern of movements performed by the person both in the spatial and temporal aspects, along the different frames of the video. One way to obtain information describing the movement of the human body in videos is to identify the joints of the human skeleton in the different frames, which can be done using 2D pose estimation algorithms in images. Currently, there are very effective and efficient algorithms available, capable of detecting the joints of the human body and returning their coordinates in the images. Allied with this, there has been a great evolution in machine learning methods and algorithms in recent years, aimed at recognizing complex patterns, inspired by biological models, with emphasis on methods based on deep and recurrent machine learning. This master's thesis aimed to propose a method for recognizing human actions in video based on skeletal joints obtained from 2D poses estimated using state-of-the-art algorithms, using deep machine learning methods and recurrent neural networks to provide more robustness to the process. The proposed method was evaluated using two public and popular databases of videos of human actions, KTH and Weizmann. The results obtained were superior to several methods found in the literature and comparable to state-of-the-art methods, with the advantage of presenting a simple strategy for extracting features from skeletal joints obtained from 2D poses.
Descrição
Palavras-chave
Reconhecimento de ações humanas, Poses 2D, Aprendizado em profundidade, Redes neurais recorrentes, Human action recognition, 2D Poses, Deep learning, Recurrent neural networks