Scientists showed monkeys a picture of a human face, scanned their brains, and then were able to construct the face just from the scan! And it was eerily accurate. Basically, the code for facial recognition in monkeys has been cracked.

Monkey see, monkey do

Two researchers from Caltech, Prof. Doris Tsao, and post-doc Steven Le Chang, found that only 205 neurons in the brain were responsible for recognizing the faces. Now that they have the code, they can reconstruct any human face from a monkey’s brain activity.

Although getting a visual of a face from scanning a monkey’s brain sounds pretty sci-fi, the way that it was done is actually quite logical. When a human (or monkey) views a face, different neurons fire to transmit the information to the brain so that it can process it. To do so, the visual information is translated into neuron activity. Think of it as a foreign language, each neuron firing is analogous to a word. When you can understand what the words mean then you can understand the language. The researchers unlocked this language and found out which information different neuron activity gives to form a face. After they cracked this language, then they could reconstruct any face from monkey brain activity. The language is reversible, and the researchers could predict which neurons would fire by showing the monkey a particular image.

The left column shows the actual face shown to the monkeys, the right column shows the image reconstructed from monkey brain activity, the predicted face). Image credits: Doris Tsao

“We’ve discovered that this code is extremely simple,” says senior author Doris Tsao, a professor of biology and biological engineering at the California Institute of Technology. “A practical consequence of our findings is that we can now reconstruct a face that a monkey is seeing by monitoring the electrical activity of only 205 neurons in the monkey’s brain. One can imagine applications in forensics where one could reconstruct the face of a criminal by analyzing a witness’s brain activity.”

The visual information in a face can be distilled into numeric values and coordinates, for example, the position and shape of your nose in relation to your cheeks and the shape of the face. The researchers chose 25 spatial positions on human faces, they also chose 25 non-spatial traits, like skin tone and texture and eye colour. They created images of human faces that had different combinations of this spatial and non-spatial information and showed them to macaque monkeys. The monkeys had electrodes were inserted into their brains to record activity from individual face cells. Face cells are neurons in particular regions of the brain that fire most strongly when a face is seen. Through the neuron activity, the researchers were able to see which neuron responded to which piece of visual information.

Macaque monkeys were used for the experiments. Image credits: Einar Fredriksen.

After knowing how the neurons responded to different visual cues, the researchers created an algorithm (the translator) that decodes any face from the firing of neurons. By showing the monkey a new face, they could re-create the face from the firing of neurons in the monkey’s brain. The faces shown to the monkeys and those recreated with the algorithm from the neuron activity were virtually identical. Only 205 neurons were enough to re-construct the faces.

The researchers can translate from neuron activity in a brain to a human face; from brain activity to visual information. In addition to cracking the code of a brain in a living animal, this study also discovered how brains recognize faces. Before, it was believed that each face cell codes one specific face. However, now we know that each cell represents one piece of visual information that combines with all of the others to form a face. Perhaps human brains also have their own code that works in a similar way. In addition to crime applications, the research could help machine learning for recognizing faces, such as photo recognition on Facebook.

Journal reference: Le Chang, Doris Y. Tsao. The Code for Facial Identity in the Primate Brain. Cell, 2017; 169 (6): 1013 DOI: 10.1016/j.cell.2017.05.011