The face of emotions

Column “PSYCHOLOGY OF EMOTIONS” by Diego Ingrassia – “The face of emotions”
for PSICOLOGIA CONTEMPORANEA – Futuro – no. 267, May-June 2018 – GIUNTI EDITORE

A famous popular saying states: ‘The eyes are the mirror of the soul’.
Another: ‘A look is worth a thousand words’.
That the face is the most expressive part of our body is a well-known fact, which explains why from time immemorial we have tried to grasp the secret behind the gaze, that set of emotions that the expressions of our face transmit, so important for our social relationships, yet not always easy to decode.

HISTORY

.
Studies aimed at trying to understand the meaning of the signals that our body is capable of manifesting are very ancient; in this regard, it is interesting to note that even a pseudoscientific discipline such as physiognomics, whose contributions we find starting from ancient Greece, has directed its attention specifically to facial expressions.
Still on the subject, famous are the studies of the Italian anthropologist and criminologist Cesare Lombroso, oriented towards establishing correlations between a person’s physical appearance and his psychological and moral qualities, studies that are significant precisely because they emanate from a vision that is totally outdated by modern scientific thought.

To find the origin of scientific studies in this field, which will lead us to discuss the instrument to which this article is dedicated, we must go back to Charles Darwin and in particular his work The Expression of the Emotions in Man and Animals, published in 1872.
Darwin was the first to theorise that the behavioural manifestation of certain emotions is not conditioned by culture and learned local traditions.
Opposing the prevailing theories of his time, with this work he highlighted how the expressions of emotions in animals and humans are innate, and thus a product of evolution.
By extending the theory of evolution to the biological substrates of cognition, Darwin laid the foundation not only for modern ethology, but for all behavioural studies and future cognitive neuroscience.

In the early 1960s, Silvan Tomkins, a psychologist working in the Department of Psychology at Princeton University, studied emotions from a biological perspective, defining ‘affect’ as the biological portion of emotion: ‘A pre-programmed, genetically transmitted mechanism’.
Drawing on Darwin’s studies, he theorised that each affect corresponded to a precise stimulus, which manifested itself through a precise facial expression, before possible cognitive processing by the subject.
He organised affects into categories, each of which was linked to a specific physiological response.
He initially identified 6 pairs of affect, later to become 9, consisting of an affect of medium intensity and one of stronger intensity, such as anger and rage.

This brings us to 1967, when Paul Ekman, a young psychologist who had been a pupil of Tomkins for a while, decides to leave for Papua New Guinea, where he joins an ethnic group, the Fore, isolated from the rest of the world, with the intention of verifying the theses enunciated first by Darwin and then by Tomkins.
The experiments conducted by Ekman on that occasion, and then repeated on other trips, demonstrated the universality of facial expressions relating to some of the main emotions.

Paul Ekman’s studies thus made it possible to discover that some major emotions – anger, fear, happiness, sadness, contempt, disgust and surprise – are represented by the same facial muscles in all peoples of the world.
This important discovery has allowed further study in this field, with the aim of establishing more precisely the correlation between emotions and facial expressions.

The FACIAL ACTION CODING SYSTEM

The human face can make more than 10 000 different expressions, many of which have no special meaning; they are simple muscular actions without any correlation of meaning.
Others, however, are decisive and meaningful from an expressive point of view.
How can we learn to recognise them?

So here we are, after the brief historical excursus, with the Facial Action Coding System (FACS).
FACS is the coding system for facial muscle movements created in 1978 by Paul Ekman and Wallace V. Friesen.
It is the first atlas of the human face that includes a systematic description (text, photographs and video footage) in order to measure facial movements in anatomical terms, breaking them down into individual movement units called ‘action units’.

‘Emotion algorithm’ is a coherent definition for a method that allows any movement of the human face to be identified in a purely descriptive manner, thus free of any possible interpretative inference.
To achieve their goal, the authors referred exclusively to the analysis of the anatomical basis of muscular movements of the face.
Each of these movements is the result of the single, synergetic action of facial muscles; in this regard, FACS takes into account the way in which each facial muscle acts in visibly modifying the configuration of the face.
The authors also made use of Duchenne’s technique of stimulating different bundles of muscle fibres with electrodes in order to reflexively determine the outwardly observable kinesic effects.
The existence of a functional identity between muscular action and related emotion is also demonstrated by the fact that the voluntary activation of certain facial muscles can generate an emotion. Ekman and Friesen found this effect when they built the FACS; after hours spent performing thousands of voluntary facial expressions, they realised that simulating an expression, such as sadness, with their face alone induced all the physiological and psychological reactions related to that emotion.

The FACS facial expression coding, analysis and intensity measurement system has emerged as the most effective and comprehensive tool, widespread in the scientific community and used in FBI and other security-related training programmes, the subject of much applied research and publications.
A FACS-certified coder is able to study every conformation assumed by the face, as a starting point for the analysis of facial expressions and their decoding, regardless of their emotional function.
But let’s leave the word directly to Paul Ekman about this tool for a moment:
“The coder using FACS dissects each observed expression, breaking it down into the individual units (AU – Action Units) that produce the movement.
This is done by viewing repeated, slowed down and stopped recordings in order to determine which unit – or combination of units – can cause the observed expressive change.
The tabulation of a facial expression consists of a list of the various units that produce it.
The precise duration of each movement, its intensity and any bilateral asymmetries are also determined.

The AREAS OF APPLICATION

Emotions activate involuntary circuits, which is why it is very difficult to hide an emotion; some muscles are activated anyway, even if only for a brief moment.
These are facial micro-expressions: manifestations that last for a very short time, between one fifth and one twenty-fifth of a second, of which we are hardly ever aware, but which leak out valuable information for those able to read such automatisms.
The FACS-certified coder is an expert in analysing people’s facial expressions (live, or from observing photos or videos of people’s faces in motion).
The applications of this tool are many; in the scientific field it is used in psychopathological research and in ethology.
The FACS method is used in the drafting of validable technical reports (e.g. in the psycho-forensic field).
Other applications are linked to the need to map, in order to provide useful indicators in a short time, a large number of people: think, for example, of the public safety sector in sensitive and crowded places.
In short, in all contexts where understanding relationships, communication and emotions, FACS becomes not only necessary, but even a priority.

A further, singular field of application, which will allow us to make some important and conclusive considerations, is that of the processing and development of digital animations for filmmaking.
The database of facial expressions used by Disney-Pixar in its most recent cartoons was structured thanks to FACS and the valuable advice of Ekman himself.
This last example allows us to highlight a crucial aspect with regard to the future development of technology and artificial intelligence.
Numerous companies are investing in software capable of reading and decoding facial expressions: we could compare the current moment in history to the time when we invented calculators to replace the human mind in performing complex mathematical operations.

But interpreting the human face is not, or at least not yet, a purely calculating operation; the algorithm of emotions still needs human guidance.
The present space does not give us the opportunity to go into this in depth, but we can point out that Ekman himself is convinced that humans are still the most reliable ‘tool’ in the analysis of facial expressions, since they can contextualise and interpret human facial expressions better than software.
When we rely completely on software, we discover that behind the numbers and statistical graphs lie potential errors in the interpretation of behaviour.
Speed in obtaining the result is not always rewarded by the validity and reliability of what we are looking for.
Therefore, employing a FACS expert capable of advising on the data captured by the software exponentially increases the validity of the results for the intended purposes.

Understanding emotions, in short, still seems to be a very ‘human’ affair.
But here we stop – no one, for now, can tell us what the future holds.