As part of the visual part of ARIA-VALUSPA, a new facial point localisation system has been developed, which builds on a novel theoretical framework, and attains state-of-the-art results in annotated benchmarks.
The facial point localisation system is the very first step in the visual part of ARIA, which allows it to know where the user’s face is, and where the main facial parts, such as the mouth, the eyes, or the eyebrows, are. It serves to further process these parts and detect subtle muscle changes, known as Action Units, which are directly linked to users’ emotions.
The problem of facial point localisation has been a research topic for the last 20 years, but it was in 2013, with the appearance of Cascaded Regression, that the field experienced a major break-trhough. Cascaded Regression builds on linear regression, and consists of a set of models that are used to tailor an initial guess of where the points might be toward where these should be. In this sense, each of the models (learnt by linear regression) just take some local information from a given image, around the input points, and then estimate where the points should be moved to be better located. This new estimate is then forwarded to the subsequent model (hence the cascade).
However, there are some flaws that cannot be directly tackled by existing Cascaded Regression methods, the most important being the incapability of performing incremental learning in real-time. Basically, the goal of incremental learning is to incorporate, to each of the models, all possible information from the subject being tracked, in order to reinforce them for any potential future frame that would require such information. The research conducted to date in the field was clearly showing the importance of it, but none of the existing methods in Cascaded Regression were capable of doing it in real-time.
Thus, as part of the research conducted by the University of Nottingham team, a new method was proposed replacing the traditional linear regression, which was coined Continuous Regression.
Briefly speaking, Continuous Regression is a mathematical solution that applies Functional Regression concepts, and permits to gather all the infinite points surrounding the real-locations into each of the models. Aside the theoretical contributions, Continuous Regression enabled for the first time the use of incremental learning under the context of Cascaded Regression.
The proposed incremental Cascaded Continuous Regression (iCCR) is the first tracker to date incorporating real-time capabilities, attaining state-of-the-art results in an extensive annotated dataset. Its impressive results were recently published in the 14th European Conference on Computer Vision (ECCV’16), which is one of the top-tier conferences in Computer Vision.
The system has been integrated into AVP 2.1 where it is part of eMax, the main part of the visual system of ARIA-Framework, and runs at more than 30fps. In addition, a specific website gathering the research conducted in Continuous Regression was launched, in which the associated papers are accompanied with MATLAB code, allowing the use of the tracker as a tool, and enabling further research on still open topics.