| AI base of science and technology (rgznai100)


Participate in | Zhou Xiang, reason_W, shawn


With the release of the iPhone X, face recognition using deep learning is expected to become increasingly standard on smartphones. In addition to identification, however, there has been a spate of recent studies examining whether facial swiping can predict personality and even behavior.

At the end of 2016, Professor Wu Xiaolin and his doctoral student Zhang Xi from Shanghai Jiao Tong University published a paper, Automatic Crime Probability Inference Based on facial images. The study suggests that, with learning, machines can tell who is a criminal from a photo and who is a law-abiding citizen with more than 86 percent accuracy.

The paper caused a stir in the AI community:
Can facial features really be used to predict behavior and personality? Is this research really not discriminatory?

In a recent paper by Stanford University assistant professor Michal Kosinski and graduate student Yilun Wang, Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. It was also highly controversial.

The study found that deep neural networks outperformed humans on the task of “identifying homosexuality” with 61 percent accuracy in men and 54 percent in women.

In addition, the typical facial features most likely to be gay men tended to be more feminine, while lesbians tended to be more masculine. Generally, men have a wider jaw, a shorter nose and a smaller forehead; Gay men, on the other hand, have narrower jaws, longer noses, larger foreheads and less facial hair. In contrast, lesbians generally have more male-like faces (wider jaws and smaller foreheads) than heterosexual women. What’s more, gay and straight people do grooming differently.

The study, published in a prestigious psychology journal, has been met with unprecedented criticism, with academics calling the authors names and even threatening emails.

“From Max: On Sep 10, 2017, at 00:06


hello



I just finished your deep learning project on detecting human sexual orientation. I think such a study should be banned. A person’s sexual orientation should be his or her private matter.



You must know that in some countries homosexuality is a crime. So I think you’re a homophobic bastard who supports the murder of gay people. If not, please destroy all work related to this topic, otherwise, I hope someone will kill you, because your work will cause many people to suffer and even die.



Please take up the knife and have a good time!



Best wishes, Max * * * *”

The authors of the paper responded to the “death letter” by saying:

“Dear Max,



You say you read my project, but do you really understand it? Before you send me to my death, would you take a moment to actually read what you wrote to me about wanting another man to die. Judge people based on hearsay, whether you’re LGBTQ or not. LGBTQ=lesbian, gay, bisexual,transgender, queer, you shouldn’t say it.



I would be honored if you actually read my project and would like to provide your thoughts/comments. And I really cherish it. And if, after reading it carefully, you still want me to write my own, then I might take such a valid request more seriously.

You can find the file here: https://osf.io/fk3xr/

You can also start with my notes:

https://docs.google.com/document/d/11oGZ1Ke3wK9E3BtOFfGfUQuuaSMR8AO2WfWH3aVke6U



Warm wishes, Michal”

In order to dispel the doubts of the outside world, the author of the paper responded to all kinds of criticism onlineAI science Base has translated the original text without changing its original meaning. After reading these replies, you may have a deeper understanding of the author’s research intention and results, and even some understanding of the causes and manifestations of homosexuality.

I. Summary of research results

We didn’t create a privacy invasion tool. We simply looked at existing technologies that are already widely used by tech companies and government departments to determine whether they pose a risk of violating the privacy of LGBTQ people.

Disturbingly, we find that these technologies do carry such risks.

Our work is really limited: we only looked at people who identified themselves as gay or straight. However, these limitations do not invalidate the results of the research or its core message: widely used technology poses a threat to the privacy of LGBTQ people.

I’d like you to consider the evidence before refuting it.

What are our main findings?

In seven studies we conducted, we demonstrated an algorithm that could accurately detect a person’s sexual orientation based on their face. Given two test subjects: gay men and straight men, or lesbian women and straight women, our algorithm correctly distinguished gay men from straight men 91 percent of the time, and lesbian women from straight women 83 percent of the time.

Mammography is only 85 percent accurate, and modern diagnostic tools for Parkinson’s disease are only 90 percent accurate.

Our dataset consisted of 35,000 gay and straight images from a publicly available dataset where users themselves had labeled their sexual orientation. On a subset of images, our algorithm achieves unprecedented accuracy. We made sure that the algorithm’s predictions were independent of age and race.

We also tested a separate sample of Facebook profile pictures with similar results.

By contrast, human judgments are no more accurate than random guesses. We think this is another example of AI outperforming humans. The study was peer-reviewed and eventually published in the Journal of Personality and Social Psychology, a leading Journal in the field. In addition, more than a dozen experts in the fields of sexology, psychology and artificial intelligence reviewed the draft paper before we formally sent it out for peer review. The study has also been approved by the Internal Review Board.

What features are used to predict orientation?

The classifiers use fixed and transient facial features. Homosexuals and heterosexuals not only have different facial shapes, but also different expressions and grooming styles.

You must be wrong – this is pseudoscience!

We get a lot of that feedback. Frankly, we’d be happy if our results were wrong. That way, humanity will have one less problem and we can go on to writing bestselling books about how smiling makes you happier.

What are the privacy implications of these findings?

It’s true that algorithms that predict people’s sexual orientation based on their faces pose serious privacy concerns. The ability to control when and to whom an individual’s sexual orientation is revealed is therefore essential not only for human well-being but also for human safety.

In some cases, the disclosure of an individual’s sexual orientation can be life-threatening. Members of the LGBTQ community continue to experience physical and psychological abuse from the government, neighbors, and even family members. Many countries have criminalized gay sex, and in some places, people who engage in gay sex can be sentenced to death.

Were the authors worried about publishing these results?

We were genuinely disturbed by these results and spent a lot of time considering whether to make it public. After the results were published, we received warnings, which we didn’t want.

However, recent news reports suggest that governments and businesses are already using tools to judge intimacy from faces. Images of billions of people’s faces are stored in digital and traditional archives, including dating platforms, photo-sharing sites and government databases. By default, profile pictures on Facebook, LinkedIn and Google+ are public. CCTV cameras and smartphones can take pictures of someone’s face without permission.

We believe there is an urgent need to make policymakers and the LGBTQ community aware of the risks they face. Technology companies and government agencies are well aware of the potential of algorithmic tools for computer vision. We believe that people should be aware of these risks and take appropriate precautions.

Until the results are published, we ensure that our work does not provide any benefit to those who might violate the privacy of others. We used a lot of off-the-shelf tools, open data, and standard methods known to computer vision practitioners. We didn’t create a privacy invasion tool, but we wanted to show that basic methods that have been widely used can pose serious privacy threats.

Why study the link between facial features and personality?

As mentioned earlier, this has important privacy implications. People and policy makers should be aware of the risks they face and should have the opportunity to take preventive measures.

The connection is also interesting from a scientific point of view. Identifying links between facial and psychological features can help us understand the origins and nature of a wide range of psychological, biological and cultural phenomena. Otherwise, many of the factors that can be easily estimated from human faces — such as prenatal and postnatal hormone levels, developmental history, health, environmental factors and genetics — will be difficult to measure. So linking facial features to other phenomena can help us generate a lot of hypotheses that can be explored using other scientific methods.

What are the underlying mechanisms that link personality traits to facial features?

There are three. First of all, personality can affect the appearance of a person’s face. For example, women who are more extroverted tend to become more physically attractive as they age.

Secondly, facial appearance can influence one’s personality. For example, good-looking people get more positive social feedback and thus tend to become more extroverted.

Third, many factors can affect both appearance and personality. These include prenatal and postnatal hormone levels, developmental history, environmental factors, and genetics. For example, testosterone levels can significantly affect behavior (such as a desire for power) and facial appearance (such as facial width and hair).

What explains the link between facial features and sexual orientation?

Normally, researchers use the widely accepted theory of prenatal hormones (PHT) to predict the link between facial features and sexual orientation. According to PHT, because androgens are responsible for fetal sexual differentiation, male foetuses develop homosexual orientation because they are underexposed to androgens and female foetuses are overexposed to males. Because the same androgens are responsible for gender anotherism in the face, PHT predicted that homosexuals generally had gender-atypical facial morphology. In other words, gay men tend to have more feminine facial features, while lesbians tend to have more masculine facial features.

Prenatal male hormone levels also affect the sexual differentiation of fetal behavior and orientation in adulthood. As a result, PHT predicts that homosexuals will generally choose gender-atypical facial grooming, expression, and grooming.



Figure 1: Composite faces and typical face profiles generated by typical faces/profiles classified as most likely to be gay or straight.

Consistent with the predictions of PHT theory, the faces most likely to be typical of gay men (see Figure 1) are more feminine, while lesbians are more masculine. Generally, men have a wider jaw, a shorter nose and a smaller forehead; Gay men, on the other hand, have narrower jaws, longer noses, larger foreheads and less facial hair. In contrast, lesbians generally have more male-like faces (wider jaws and smaller foreheads) than heterosexual women.

The sexual atypia of gay faces is not just morphological. Lesbians tend to wear less eye makeup and less revealing clothing (with a lower neckline), and have darker hair — less feminine dress and style. In addition, straight women smile more often than lesbians.

In addition, the theory supports the relationship between baseball caps and masculinity in American culture: both heterosexual men and lesbians seem to prefer baseball caps (see the shadow on the forehead in figure 1; Manual detection of a single image proves the validity of this conclusion.

What explains the accuracy of the algorithm?

How accurate is the classifier? Interpretations of classification accuracy are important, and the conclusions are often counterintuitive!

A hypothetical sample of 1000 men, including 70 gay men, were evaluated using a classifier with an accuracy of AUC=.91 (compared with the male face image classification experiment in this study (5 images per person).

Although the classifier cannot point out which detection object is gay, it can mark the probability that each detection object is gay. It’s important to decide what the cutoff point is — or what the probability is — to label someone as gay.

If you want to choose a few gay as sample and keep very small error rate – will be marked with the highest probability of a few objects gay, so you can get high accuracy (for example, a small number of objects) marked as homosexual, but at the same time recall (recall) will lower (for example, “miss” will be many gay men). When you expand the range, you will ‘detect’ more gay men, but there will also be more straight men who are mislabeled as gay (a so-called ‘false positives’). In other words, the pursuit of high accuracy will lead to a lower recall rate and vice versa.

Back to the sample of 1,000 men, including 70 gay men. If 100 men were randomly selected from this sample, only seven of them would be expected to be gay — with a random sampling accuracy of 7% (seven out of every 100 men in the sample).

Classify with classifiers. According to the classifier, 47 of the 100 men with the highest probability of being gay were gay (accuracy = 47/100 = 47%). In other words, classifiers can improve the accuracy of random sampling by nearly seven times.

We can further improve accuracy by shrinking the subsample. Of the 30 men with the highest probability of being gay, 23 were gay (accuracy = 23/30 = 77%; Recall rate = 23/70 = 33%), which was 11 times more accurate than random sampling (77/7% = 11). In the top 1% subsample (the top 10), nine gay people were identified (90 per cent accuracy) : a 13-fold improvement in the accuracy of random sampling. But achieving such high accuracy comes at the cost of a low recall rate: only 13% (9/70 = 13%). To improve accuracy, you must sacrifice a certain recall rate.

You must be wrong – this is pseudoscience!

Like any scientific study, our study may have its imperfections. To that end, we have listed some of your concerns and answered them:

“Surely you are wrong; The subjects in this experiment were all white.”

Although we sought to obtain a more diverse sample, this study was limited to caucasians within the United States.

This does not invalidate the conclusions of this study. This study proves that you can tell gay people from straight people.

Although this study does not prove that this is true for other ethnic groups, we do find that it is possible. The same biological, evolutionary and cultural factors that drive differences between gay and straight people are likely to affect other races as well.

“Surely you are wrong; Bisexuals were not considered in this analysis.”

Yes, we didn’t explore whether you could predict bisexuality from a person’s face.

But that does not invalidate our conclusions. We’ve still shown that we can tell gay people from straight people. Some objects classified as straight or gay may actually be bisexual. However, correcting such errors might improve the accuracy of the classifier.

Importantly, not considering bisexual or transgender people does not mean we deny their existence.

“It must be wrong; The sample was members of dating websites who were openly sexual.”

This is a reasonable limiting factor, which is discussed in detail in our paper. Indeed, there seems to be a problem with the image data collected from dating sites, where sexual orientation information is particularly obvious, but this study doesn’t stop there.

First, we used an external sample of Facebook images to test our classifier as accurately as we could use dating site images. This suggests that Facebook profile images convey just as much information about sexual orientation as images on dating sites.

Second, we asked study participants to judge sexual orientation based on their faces. The participants were not much better at judging images carefully standardized in the lab than in previous studies. This suggests that the graphic sexual orientation information used in this study isn’t particularly obvious — at least, not to humans.

Finally, the deep neural network used in this study has been specially trained to learn only fixed facial features that cannot be easily changed, such as the shape of face elements. This helps reduce the risk that the classifier will find some superficial difference in the images of gay and straight faces in the study that has nothing to do with faces.

“Surely you are wrong; It is well known that there is no correlation between facial features and personality traits.”

Unfortunately, there is no basis for this claim.

Numerous studies have shown that humans can judge other people’s political views, personalities, sexual orientations, qualities and other traits, but with limited accuracy. Poor accuracy in these judgments does not necessarily mean there are no obvious signs of these traits on the face, but rather that humans may not be able to see or interpret them.

“You must be wrong. Your classifier must have selected something unrelated to facial features in its prediction.”

We also thought about a lot of related things. And we very much hope that future research will be able to more convincingly prove or disprove the possibility of using faces to predict orientation. Of course, we have done a lot of our own work to improve the rigor and persuasiveness of the study.

  • First, our model trains specifically on fixed features of the face that are hard to change, such as the shape of facial elements. The deep neural network we use is also trained for an entirely different task: recognizing the same person from an image. This helped us reduce the risk of surface differences that the classifiers found between the images of gay and straight faces used in the study, differences that weren’t even related to faces.
  • Second, we performed a secondary validation of the results on external samples.
  • Third, we studied which elements of the face image could be used to predict orientation to ensure that these elements were indeed facial features (and not other factors). As you learned in the paper, even when all the visual information was removed, the classifier could still make fairly accurate predictions based on the contours of the face.
  • Fourth, we only had the classifier detect the face area and removed the background area outside the face from the image. We also checked to make sure the classifier was focusing on facial features rather than background when making predictions. The following thermal map (seen in Figure 3) clearly shows that the classifier detects a portion of the facial area (red) rather than the background (blue)



Figure 3: A thermal map shows the extent to which different given portions of the labeled image can alter the classification results.

Color scales ranging from blue (unchanged) to red (substantially changed) indicate different results. We use 2D Gaussian filtering to smooth the color-coded blocks.

Finally, and perhaps most importantly, the differences the classifiers found between gay men and straight men in faces are consistent with what the prenatal hormone theory, a widely accepted explanation for the origins of sexual orientation, would predict.

“Surely you are wrong; Your inquiry
It turns out that gay people tend to be gender-specific — but I know a lot of gender-specific gay men and lesbians!”

We also know that there are a lot of very masculine gay men and a lot of very feminine gay women. It’s like, we know there are a lot of older men, but that doesn’t disprove that women live longer. The fact that gay men have more feminine facial features does not mean that all gay men are more feminine than straight men, or that there are no very masculine gay men (and lesbians, too).

The differences we observed between femininity and masculinity in our study are subtle and show up in many different facial features; Though imperceptible to humans, these differences are already noticeable to more sensitive algorithms.

“Surely you are wrong; Many of the participants in your experiment must have lied about their sexual orientation!”

Indeed, some of the participants who told us they were straight were actually more likely to be gay (and vice versa). However, we believe that those who voluntarily post profiles on dating sites to find a partner have little incentive to lie about their sexual orientation.

Of course, if some of our participants did lie about their sexual orientation, then catching them in their lies would most likely further improve the accuracy of the classification.

“Surely you are wrong; The only reason is because gay people care more about their image or take better photos!”

It’s easy to believe that gay men have better hair and beards. As we discussed in our paper, gay and straight people do differ in grooming.

However, they are also strikingly different in form. Our algorithm was more than 70 percent accurate for gay men who only had facial contours, and more than 60 percent accurate for lesbians.


Original address:

https://docs.google.com/document/d/11oGZ1Ke3wK9E3BtOFfGfUQuuaSMR8AO2WfWH3aVke6U/edit#