Automated System For Detecting Asymmetric Spontaneous Smiles

Asymmetric smiles show contempt, disapproval, doubt and defiance. When we see a guy not able to solve a simple mathematical equation or not aware in which foreign country the prime minister’s will be; only one corner of lip goes up while smiling on his lack of common knowledge. This asymmetry indicates a strong negative emotion.

To develop a system detecting spontaneous asymmetric smiles as people watched online videos, Senechal et al. did a study including one-sided and asymmetric facial expressions. The right facial nerve is independent of left facial nerve. When it comes to lower part of face, the muscles are contralateral which makes us easier to perform asymmetric expressions such as smirk. As muscles of upper face are bilateral, performing asymmetric movements is difficult. Try to raise only one eyebrow like the famous eyebrow-raising of ‘The Rock’.

Researchers collected and labeled 2265 videos of spontaneous data of clips ranging from 30 seconds to a minute. They also collected 200 posed videos in participants were asked to pose various asymmetric facial expression with slight variations like tilting their head. The template matching technique was used to identify asymmetric smiles in spontaneous data clips.

For each frame of video, the face was located, scaled, cropped and flipped around vertical axis to train and test. It was found that training with combination of mixed and spontaneous data gave best results.

After training and testing, they achieved 69% precision in detection which can be interpreted as only 1 false detection of asymmetric smile in every 85 videos. This system can be used for automatic detection of spontaneous asymmetric smiles. This provides valuable understanding of how people take part while watching videos or films. The emotions they convey while watching a fiction or real film gives insights about human nature through different aspects.

-----------------------------------------------------------------------------------------------------------------------

Reference - 

Researchers - Thibaud Senechal, Jay Turcot and Rana el Kaliouby. Affectiva Inc., Waltham,

MA, USA.

Research paper name - Smile or Smirk? Automatic Detection of Spontaneous Asymmetric

Smiles to Understand Viewer Experience.

Published in: Automatic Face and Gesture Recognition (FG), 2013. 10th IEEE International Conference.