Human Action Recognition Based on 2D Poses and Skeleton Joints

Abstract

With the growing capacity of current technologies to store and process data at extremely fast speed, video processing and its diverse applications have been studied and several researches have emerged using videos as objects of study. One of them is human action recognition, which seeks to identify in a given video what actions the people present in it are performing, whether for recreational purposes or for monitoring public places for people’s safety. Being able to extract the necessary characteristics to perform the action classification is a complex task, because, unlike traditional image classification problems, the recognition of human actions requires that the solution works with spatio-temporal characteristics, which represent a pattern of movements performed by the person in both the spatial and temporal aspects along the frames that make up the action. One way to generate information that describes human movement is to identify the skeleton joints and limbs and use them to perform the classification, and the extraction of this data can be done by using 2D pose estimation algorithms, capable of tracking parts of the human body and returning their coordinates. This paper proposes a method for human action recognition that uses skeleton joints and limb information as features and a CNN together with an LSTM network for classification. The method was assessed on a public dataset (Weizmann dataset) and obtained superior recognition rates when compared to other action recognition methods of literature.