What do we actually know about Face Recognition Technologies?
You deal with face recognition technology almost every day, but you hardly ever really thought about it, do you? The first thing that comes to mind is a face ID to unlock your mobile phone, which obviously from the title uses face recognition technology. But not only your phone and various applications now “recognize” your face: on social networks, when you tag your friends in different photos, airport checks, cameras in banks, and even street cameras are actively using these technologies. For example, the police can already compare photos in their database with photos from a street camera and find criminals. And every year it develops by leaps and bounds before our eyes. Remember how in the beginning, just a few years ago, the iPhone could not unlock the phone if you were wearing wide-rimmed glasses or too bright makeup?
After just a couple of years, video intercoms with face recognition are already available to us! Just imagine, uploading a photo of an unloved neighbor and you are never "at home" for him.
How did it get so widespread?
This is due to progress in three technical areas: the emergence of the most powerful graphics processors or GPUs, big data, and the development of neural networks.
The latter is the basis of modern artificial intelligence, which made it possible to process a huge amount of information - your photos that you upload to Google, Facebook, Instagram, etc., and further training. These photos are used to train neural networks, which contributes to better face recognition by artificial intelligence. Of course, such work cannot be done without serious hardware - cool ultra-fast chips, and graphics processors. Let's not forget about social networks. According to multiple statistics of users of various social networks in the world in 2021, more than 4 billion people use social networks and upload their photos. That's almost half of the world's population!
One, two, three, four, and a decade later we have a face recognition system integrated everywhere. It seems that science fiction books have long ceased to be such, rather they are gushing with ideas for the development of new technologies, like facial recognition technology, for instance.
"Big Brother is watching you"
In China, facial recognition technology is actively used to control and monitor people. The police can see the movement of specific people of interest to them through cameras on the streets, at airports, banks, etc. According to BBC News since last year, the Chinese government has been testing video cameras using artificial intelligence and facial recognition technology to detect human emotions in Xinjiang. Many human rights activists say that this violates the rights of people living in this territory and that such experiments are politically motivated to total control over the Uighurs, who make up almost half the population of China's largest autonomous region, almost 11 million people according to the official census in the 2020 year. But even in general, the Chinese are striving to create one large network of video control over the population and are extremely interested in the development of related technologies.
In the UK, the Metropolitan Police are trying face recognition technology to find criminals not only on the streets of the city but also in stadiums during sports matches, music festivals and concerts, and ceremonial events.
This technology was also used at all the 2020 Summer Olympic Games facilities in Tokyo, which were postponed to 2021 due to the coronavirus pandemic. Instead of the usual passes and ID, they used a contactless identification method for greater security in modern conditions.
How does it work?
The technology is usually based on the use of two trained neural networks.
So, the first step is discovery. The computer must find the face in the image. On a huge number of photographs that have people's faces in different positions, the first algorithm learns to find these faces. That is, each time the neural network must evaluate where there is a human face in the photo. By repeating this action repeatedly, the algorithm improves over time.
The second step is recognition. Using a second neural network, the computer trains to distinguish one face from another. For example, he places the so-called anthropomorphic points on the face, and these points determine the distance between the eyes, mouth, and nose, the height of the forehead, etc., there can be thousands of parameters. Other systems may not use points, but a more abstract algorithm for finding lighter and darker areas of the image, for instance.
Real-time video surveillance software typically scans individual shots of video taken in crowded places, such as subway entrances. And according to the steps described above, it determines the faces in the image, calculates the vectors for each, and then compares it with the face vectors in the database. For similar faces, the neural network gives several matches. Depending on the given threshold of coincidences, the police get their result. In the UK the threshold is usually 60%. The higher it is, the more likely it is to avoid false matches.
Police can also use facial recognition technology in reverse order and trace the detainee's path to the crime scene. Upon arrest, upload a photo of a potential criminal to a computer and use this technology to follow the camera recordings of his movement to the crime scene.
Is it all so smooth?
At first glance, the accuracy of facial recognition technology is striking. According to independent research by the NIST, the error rate dropped to 0.2% between 2016 and 2018. Tests show that the technology system has become dozens of times better at finding spitting images in a database of 12 million photos of people. It is all to blame for the training of neural networks and the emergence of a new generation of GPUs.
But all these rosy results are based on clear photos of different people, which are compared with perfect photos from the database. However, in real life, most often the pictures are not so clear, people can turn away from the camera, and their face can be covered with a hat, scarf, and big dark glasses, after all, people can be much older than their photos in the database. All these factors affect the recognition result, significantly reducing statistics.
It is also problematic for neural networks to recognize twins.
The bigger problem is bias.
A certain racial and gender bias in facial recognition systems in the United States has caused a lot of discontent among the people. The fact is that for training algorithms, mainly photos of people of the titular race and nation are used, and most of the photos used are male. The problem arises in recognizing African Americans, Asians, and Hispanics.
In 2018, MIT conducted a study whose results showed that the algorithms of IBM, Chinese Megvii (FACE ++), and Microsoft all too often misidentify black women (34.7%) than light-skinned men (0.8%).
In the same year, ACLU conducted its testing of technology from Amazon, where it mistakenly identified photos of 28 members of Congress as matching images in the US police database of violators of the law. Also, the recognition of the faces of African Americans and Hispanics was not correct and not proportional. After that, Amazon made a statement that the settings of their face recognition system were not configured correctly by ACLU, which led to this result.
Additional shortcomings of the system were identified by researchers from Cardiff University, who tested the results of the NEC NeoFace system. It froze and failed during adverse weather conditions, a large crowd of people in the shot, in the late afternoon, when the lighting changed. At this time, the light sensitivity of the camera increased, and, accordingly, the level of "noise" in the photograph also increased greatly.
For two days of the experiment, NEC NeoFace found 2900 matches, 2755 of them were false positives.
This study also showed that there are people that can be randomly lookalike several people at the same time.
Scanning a crowd of people during a rugby match, the NEC NeoFace facial recognition system identified the same woman on the police watch list 10 times. Although this woman has never broken the law.
Who is on top?
The top 3 countries that actively use face recognition technology are China🇨🇳 (626 million as of 2020), the United States🇺🇸 (about 80 million), and Russia 🇷🇺 (almost 15 million). Japan, Israel, and Europe are also actively using this technology.
Although it is the Chinese approach that is most similar to Orwell’s scenario: cameras with a face recognition system are already installed everywhere, and during the pandemic, temperature sensors were added to this technology to identify sick people during trips. Punishments for the spread of the virus are imminent.
In the US, this technology is being used by police more to find perpetrators through CCTV footage, but the trend of scanning crowds in real-time is slowly gaining popularity.
As a result of a study by the US National Institute of Standards and Technology, the Russian technology NTechLab was recognized as one of the best. For the first time, this system was launched in Russia during the World Cup in Moscow. This system was used more during the pandemic in 2020 and during rallies in 2021, when using newly equipped cameras, police officers found people even at the entrance to the subway and came to people's homes to hold them accountable for violating the law.
What about the laws?
The situation is still unclear and therefore the use of facial recognition technology around the world is controversial. In the UK there are no laws at all allowing the police to use this technology, nor is there any state policy regarding this issue. It turns out that the police can independently decide when and how to use photos from CCTV cameras. This raises serious concerns and is contrary to human rights law, according to a University of Essex report on an independent review of London police's use of facial recognition. Researchers have found that too many people on the street are stopped by police by mistake and have speculated that the system is being used to track people not wanted by the court, creating watchlists in defiance of a 2012 Supreme Court ruling that it is illegal to keep photographs of innocent people. It is concluded that since that time, law enforcement agencies have been accumulating a database of a couple of tens of millions of people who have never violated the law. Still, photos from databases and photos extracted from social networks are used in the face recognition system.
In the private sector, there is no control at all: photos of people from surveillance cameras in shops and enterprises can be entered into private databases and watchlists and transferred to third parties.
EU authorities have shown their willingness to support upcoming bills to ban the use of facial recognition systems in public places across the EU. In October last year, a corresponding resolution was adopted, with the help of which deputies expressed concern that the use of AI by law enforcement agencies is fraught with high risks of discrimination, invasion of privacy, problems of personal data protection, and may affect decision-making and could be a threat to even freedom of speech and information. The resolution is not binding, but it is a clear expression of its position by the European Parliament.
In the US, the story is no better. There is no single federal law governing the use of this technology. Only a few states have laws regarding the use of facial recognition technology by law enforcement. This means that while in the city of San Francisco the law prohibits real-time facial recognition, in the state of Arizona, for instance, any provincial sheriff's office has put photos and documents of every county resident on their watchlist.
How other biometric technologies are doing?
So far, face recognition technology rapidly developed and has already reached omnipresence. Scientists and various organizations find other biometric technologies that could uniquely identify a person.
The police are interested in such biometrics as gait because the analysis can be carried out without the participation of the person himself. Algorithms can identify a unique gait style that shows differences in genetics, habits, personality traits, and even social backgrounds of the person being studied.
Facial recognition problems caused by facial masking with scarves, masks, and unusual facial expressions could be addressed by analyzing the distance between pores in the skin. This method is still under development, but the prospects, according to scientists, are quite promising. With this method, they hope to also solve the problem of identifying twins.
Not new, but also gaining popularity in voice recognition system, which is already often used in many banks. Human voice recognition is different from long-standing speech recognition. Using a new method of identifying a speaker, the algorithms establish unique acoustic structures that are created by speech habits and the human vocal tract. As you might guess, everyone has their own characteristics of speech, and this method helps to uniquely identify each person.
Even more applicable than above is the recognition of veins under the human skin. Special scanners can see "maps" of blood vessels in the eye, hand, and even finger. Such data is difficult to fake.
What does the future hold?
Experts believe that facial recognition technology will become mainstream in the coming years. Everything is gradually moving towards a greater loss of privacy.
Dubai-based NNTC and US-based Vuzix are jointly developing facial recognition glasses. Reportedly, the glasses will be equipped with an eight-megapixel camera, which will scan the faces of passers-by and search for matches with databases.
In the UK and the US, body cameras are being developed for police officers that will operate on the same principle. In the United States, a patent was even recently filed for a body camera that instantly starts recording when a suspect is detected.
Meanwhile, IT firms around the world are actively working to upgrade face recognition technology to speed up its work, solve problems with identification in photos with many people, noisy photos taken in bad conditions, images of masked faces, and so on.
The UK Home Office developed the Biometrics Strategy back in 2018. The idea was to make the human identification system more efficient by combining facial and gait recognition technologies with voice recognition technologies. But it has not yet been put into practice.
On the other side: scientists and developers are trying to fool neural networks in a variety of ways. From the development of special makeup by Gregory Bakunov from Yandex based on the anti-similarity genetic algorithm to the special sunglasses of Carnegie Mellon University researchers from Pittsburgh. However, to deceive neural networks, you need to know exactly their setups. After all, no one knows for sure how certain algorithms will be trained.