We use our faces to communicate, but our facial expressions may not always come across the way we think they do. And we may be just as wrong when reading the faces of others, a study says.
People don’t always understand when faces intend to convey feelings such as happiness, anger, fear or sadness. Different facial expressions may mean different things to different people. “What one person sees as anger, for example, another might see as fear or sadness,” Binetti said.
He and his colleagues set out to test whether the 336 study participants could agree both in creating and then identifying facial expressions. They developed what they described as a “genetic algorithm” tool inspired by natural selection mechanisms. It allowed the participants to design 3D faces over several iterations that expressed their ideas of happiness, sadness, fear or anger.
“The process was a lot like dog breeding, where a breeder can select traits to add or remove in a dog population,” he explained. “Similarly, our participants took three dimensional faces and progressively evolved them by selecting expressions that were the closest match to the one they were aiming to create.”
“We found that, despite considerable overlap between participants' expressions, each person has a unique take on how each emotion is expressed, like an emotion expression ‘fingerprint’ of sorts,” he added. “These differences carry over to people’s ability of recognizing emotions portrayed by others.”
The researchers then screened the participants with a more traditional standard emotion recognition test and found that their responses reflected the same differences as with the faces they created, and closely matched them.
“People can see expressions differently, especially when dealing with expressions of sadness and fear, which are emotions that frequently overlap,” he said. “One person might see sadness, whereas another person might see fear because they have different preconceived notions about how these emotions should look. These notions may be the result of our genes, our cultural background or social norms, or some combination of both.”
Incorrectly gauging other people’s feelings could have unintended social consequences, particularly on how people interact with others, said Tanja Wingenbach, a lecturer in psychology at the University of Greenwich’s School of Human Sciences who studies how emotions are perceived and expressed. She was not involved in the study.
“The study suggests that it could prove problematic in everyday life when people interpret facial expressions very differently from what is intended,” Wingenbach said. “Interpersonal relationships — marriage, friendships, connections with co-workers — could all be affected.”
But she also noted that, in the real world, most people use additional methods to communicate, such as vocalization and speech inflection, which can make a positive difference. “Research shows that emotion recognition is improved when we combine information from multiple channels,” Wingenbach said.
Visual interpretation works both ways, she said. “When listening to a friend talking about a sad experience they just had, we might display a sad facial expression to be sympathetic without really experiencing sadness ourselves,” Wingenbach said. “The task for us is to show our empathy in a way the other person will understand, even if we aren't exactly feeling it ourselves. That means we have to figure out the most appropriate facial expression to get that message across, which can be difficult when different people interpret facial expressions differently.”
The findings could have implications for health, legal and other settings, Binetti said. A better understanding of facial expressions could help in diagnosing or treating certain conditions such as schizophrenia, depression or autism, since people with these disorders often find it hard to recognize or display appropriate facial reactions. “People with depression have a tendency at perceiving expressions as being sad, even when these are neutral and convey no emotion,” Binetti said.
We can’t assume “a common understanding of what emotions different facial expressions reflect,” said study co-author Isabelle Mareschal, professor of visual cognition at Queen Mary University in London. “This could have important consequences for the clinical understanding of certain conditions, where people appear to have ‘atypical’ responses to a facial expression.”
These differences could also play out in courtrooms where juries must decide the outcome of a case based on witness testimony, Binetti said. “Imagine two members of a jury listening to a defendant making a claim on the stand,” he said. “The jurors might interpret the defendant’s expressions and body language in different ways, leading one to see the defendant as feeling genuine remorse, while the other might see the defendant as afraid but remorseless. These interpretations can color how they judge the defendant’s words, potentially leading to different votes.”
People can agree on which expressions represent certain emotions when they experience them in a certain common social environment, Binetti said. “As a group, we converge on a vocabulary of expressions, just like we agree on words to express concepts,” he explained. “But expressions come in all shapes and forms, sometimes more clear-cut and intense, other times more subtle and ambiguous — and sometimes people will disagree on their interpretation. The question is: what leads someone to misjudge an emotion?”
Cultural distinctions also may play a role, he said, pointing out that people from different parts of the world differ in how they picture various emotions. “Subtle but important aspects of nonverbal communication might be lost in translation,” said Binetti, who is from Italy but lived in the United Kingdom for many years before returning to Italy. “I still occasionally struggled reading British body language to work out whether a joke or a comment of mine is funny or too edgy.”
We can’t rely on reading people’s expressions to understand their intent, he said. “The key is figuring out when we can rely on this information and how to weigh it against other clues of others’ intentions and their state of mind,” Binetti said. “There is no magic formula for this. Common sense and experience go a long way in helping us figure these things out.”
