A Vital Component In Today's Security & Commercial Landscapes

A face remains the most widely used way of identifying or authenticating a person. A photo of it is on most identification documents that we carry in our wallets. A lot of information can be provided from a person’s face, clothing, and appearance, and today a person’s face has become the epicentre of the most fascinating and promising evolving forensic technology – face recognition.

In the field of biometrics, perhaps nothing stirs up as much debate as face recognition. Using technology to identify or verify a person has always been something easily understood in theory. Heavy use of such technology in Hollywood blockbusters like Minority Report has certainly helped the technology gain widespread exposure among mainstream audiences.

While humans have always had the innate ability to recognise and distinguish between faces, computers only recently gained the same ability. It is still improving rapidly today, but from the time it was first worked on in laboratories in the 1960s, the technology has advanced by leaps and bounds. In 2006, a test of several face recognition algorithms by the National Institute of Standards and Technology (NIST) showed that machine recognition has improved tenfold since 2002 and a hundredfold since 1995 1. The best algorithms actually performed more accurately than most humans can manage.

That was in 2006. Since then, face recognition has been one of the most progressive technologies in the world, thanks in part to new methods of measuring up a face.

In 2017, NEC’s face recognition technology had an accuracy of 99.2 per cent when it comes to detecting the right person at an airport passenger gate. This was confirmed in NIST’s tests that were conducted to recognise one individual at a time as they walked through an area without stopping or acknowledging the camera.

Face To Face With The Technology
In the past, many algorithms relied on 2D measurements between a person’s eyes. Despite this being a sensible method, results can be profoundly affected by the angle of view or a different facial expression. Even the smallest changes in light or orientation can reduce the effectiveness of a system, rejecting a match to any face in the database, and leading to a high rate of failure. Face recognition technology today can check on a wider number of features.

Today, face recognition technology uses advanced pattern recognition models and captures images in real time to select areas of the face with dense information values — such as the curves of the eye socket, nose and chin — to identify the subject.

These areas are all unique and do not change over time. Face recognition today can even be used in darkness and has the ability to recognise a subject at various view angles. These are just among some of the many features that have come to define the technology in recent times.

To achieve reliable face recognition of a video image, NEC developed feature-point extraction technology that enables enhanced face recognition to a level where an individual can be identified with high precision from within a group, even if their face is partially hidden, or the image is taken from different angles.

NEC’s face recognition technology also uses deep learning technologies for face matching to increase accuracy to a level where an individual can be identified by a low-resolution face image captured by a distant camera.

These enhancements helped NEC come out tops in a comparison of various vendors’ technologies in NIST’s 2107 Face in Video Evaluation (FIVE) test. NEC’s solution was both fast and accurate, two important qualities that are needed for real-time use cases.

It is also a sign of how far face recognition has come. Back in the early 2000s, several early and high-profile installations, by police and airport authorities, for example, were scaled back because of a lack of accuracy. Today, as the technology has improved, so has the acceptance.


Previously built with the vision of improving security within the law enforcement space, the breakthrough in face recognition now sees the technology being applied within a larger audience. On top of providing enhanced security today, the technology has proven to be versatile enough to be deployed across areas of financial services, entertainment, advertising and many more.

Banks, for example HSBC in London, use face recognition to enable authorised users to enter a secure vault. This method of authentication is not only easy to use, but also highly accurate and secure.

Credit cards can also be protected by the safest, most user-friendly authentication available–the owner’s face. A person paying for items at a cashier can be verified by a camera connected to a face recognition system. Pass the “face test” and the payment goes through.

Increasingly, too, companies are seeking to deploy face recognition to know customers better and provide improved service. Commercial uses, such as for customer service management, queue monitoring or business intelligence, are now benefitting from the advances in face recognition in recent years.

In London, retailers have used a face recognition solution from NEC to identify its loyal or VIP customers walking into a store. A camera picks up the image, which a computer then compares against a database. When a VIP walks in, a sales assistant is quickly alerted to provide more personalised service to him.

In Osaka, the Universal Studios theme park uses face recognition to identify annual pass holders. This allows the holders to enjoy a VIP experience without requiring additional paperwork.

These are but a few examples of what face recognition can offer in the coming years. With vast amounts of data collection, storage and crunching that is made available through the use of cloud technologies, it is going to provide even more insight for organisations interested in both commercial and security applications.

Here’s a look at four scenarios where facial recognition may be deployed:

1. Controlling Access
Comparing a person’s face with a photo ID is one of the oldest ways to verify if he can enter to a restricted place. Now, face recognition just makes that task much easier, allowing security staff at border control or even a casino to prioritise on other duties such as identifying potential offenders based on their behaviour.

In Canada, the Ontario Lottery and Gaming Corporation has been turning to face recognition to automatically recognise some 15,000 self-identified problem gamblers–and keep them away from its betting tables.

Cameras photograph everyone who walks through the front doors of a casino, which gets 40 million visits a year. A computer then compares the images against a database of individuals who have put themselves on a self-exclusion list.

Privacy safeguards are in place as well. The photographs of people who don’t match any images on the database are discarded. Those in the database are biometrically encrypted and only decrypted when a person in one of the pictures is present.

In the same way, face recognition can be used for border control. Canada, incidentally, is also one of several countries to issue e-passports with a smart chip that can be used with face recognition technology. This allows officers, possibly in future, to compare a physical person’s appearance with a digital photograph that is stored on a chip and not easily faked.


2. Finding Subjects In Law Enforcement
In the security and law enforcement sectors, the usage of face recognition is arguably the most obvious – and often accepted – to most citizens in a smart, connected city.

The technology can be used to identify, for example, known suspects who are the subject of a police investigation.

To do this, law enforcement agencies would have to not just collect thousands of images – both video and still images – but also to analyse the information within the tight time frame available to solve a case.

In the United States, the Federal Bureau of Investigation (FBI) is spending US$1 billion in a Next Generation Identification (NGI) program, which will add biometrics such as iris scans, DNA analysis and voice identification, along with face recognition, to an investigator’s toolkit.

There are several possible use cases. Law enforcement agents might try to identify fugitives or missing persons from a face recognition database. They may also do so for unknown persons of interest. Using sophisticated multi-camera technology, they may even track subjects moving from one place to another after a critical incident.

If agents already have a suspect, they may also look out for someone they’d expect to turn up at a certain place, such as a crowded stadium. Conversely, if a riot had happened at a stadium, investigators could look at the video footage to identify who the rioters were.

The technology is already there now. The challenge is to identify subjects from a fast-growing database of photos and videos. Sometimes, face recognition is one clue in a digital trail left by a subject, for example, when he makes a call or uses a credit card.

To tap on a multi-modal database of information, many law enforcement agencies are looking to cloud-based solutions, where storage and computing capabilities are on tap should an investigation scale up.


3. Identifying Customers
A relatively new use of face recognition is in the retail space. This enables companies to quickly and seamlessly identify a frequent customer, and ensure that he or she gets personalised service without having to explicitly disclose his or her identity to the shop assistant.

A camera at the store captures images of everyone stepping inside, then sends the data to a remote computer that compares them against a database of known VIP customers. Various measurements of the face are taken and checked against a template. If a match is found, usually within a split second, the sales assistant is alerted on the smartphone or iPad to provide personalised service. He or she would also be fed details such as the customer’s dress size or other preferences gleaned from previous purchases.

Already, a dozen top stores and exclusive hotels in Britain, America and Asia have been testing the face recognition technology provided by NEC. It works even when people wear sunglasses and other items that cause other face recognition technologies to stumble.

Because NEC uses a holistic method of combining various ways to detect a face, it works similarly to how a person would identify another. Just like how we may recognise a friend wearing glasses or with a slightly new haircut, the system is able to adapt to these subtle changes and correctly recognise a face.


4. Putting More Than Just A Name To A Face
Regular customers, of course, still count. Retailers can now “see” every single customer who walks into and uses that image data to estimate the typical gender and age of their clientele. All they need now is a camera, PC and a subscription to NEC’s cloud-based face recognition service.

Leveraging on NEC servers on the Internet, the service takes images sent from a merchant and does the analysis off-site, saving merchants the heavy investments required to deploy the technology themselves. The service costs 70,000 yen (US$880) a month in Japan and has been running since its launch in 2012.

The use of such face recognition technology extends beyond retailers. Large malls, for example, could accurately determine the type of crowd it attracts in certain times of a day at certain parts of a complex.

The same technology could be used in interactive advertisements, which can estimate the age and gender of a person in front of them. The messages can then be customised to fit the person, resulting in a more effective pitch.

Ultimately, with intelligence gleaned from actual customers and users, retailers and advertisers can better strategise marketing plans to target the right demographic groups.


Ensuring Privacy Is Protected
All the good technology cannot be used if organisations rolling out face recognition in their operations do not get the critical buy-in from users themselves.

Unlike fingerprints, it is easy to capture an image of someone’s face without consent. That is a double-edged sword. Face recognition is often viewed as a “contactless” form of surveillance, identification and verification, which makes many people understandably concerned, especially about preserving their anonymity in public.

In the recent years, Facebook’s controversial deployment of face recognition technology to identify people in uploaded photos and even law enforcement efforts to capture and analyse publicly available images have also met with privacy concerns.

These are key issues to address should face recognition be adopted by organisations in both public and private spaces. What is important is not just the capturing of such information, if it is in public or clearly shared by a user, but the design and governance that come with storage, access and analysis of such personal data.

The Federal Trade Commission put out a staff report in 2012 2, where it listed some best practices that organisations can put in place. For example, companies that hold databases of consumer images should design their services with privacy in mind. Should the images be no longer needed, they should be deleted. If retained, they have to be reasonably protected.

Transparency to end users is another key point. Organisations deploying face recognition in digital signage, for example, should make clear to users that they are being observed, since they may not detect a camera in the setup.

Yet another issue brought up by the report concerns social media networks. Without a consumer’s consent, they should not identify him to someone who would otherwise not be able to identify him. For example, a mobile app that lets a person identify another at a bar, with possibly added information such as his address, could raise safety risks and only be allowed with the user’s consent.



When governments and commercial organisations decide to turn to face recognition technology, a number of questions come up often. Here are four that should be asked of a vendor proposing a solution.

1. How fast, accurate or reliable is it?
Independent tests provide a good gauge of how good a face recognition system is, and NEC has regularly come up tops in tests in the United States.

In 2009, NEC’s face recognition technology was ranked number one in the Multiple Biometric Grand Challenge’s (MBGC) “Still Face Challenge”, carried out by the National Institute of Standards and Technology (NIST) and commissioned by the United States Department of Homeland Security. In 2010, NEC again achieved the highest score in the new Multiple Biometric Evaluation or MBE 2010 benchmark.

NEC’s technology is also often seen as holistic. It takes a number of methods to recognise a person, thus overcoming common issues like a unique facial expression or a pair of sunglasses blocking key features. Like how a human recognises another, our solution uses various methods to identify a person, bypassing problems that usually fool other systems.

2. Does the face recognition require specialist, expensive cameras?
Some face recognition solutions are highly accurate only when fed high-resolution images taken by expensive, specialist cameras. This not only requires a heavy investment to cover a large area, but also makes existing cameras unusable.

NEC’s solution works with a diverse range of image sources, including live cameras. The resolution required for an accurate identification is also modest, making our solution a good fit in a city with various types of camera systems already in place.

3. Does the vendor provide a way to store and analyse the images on-demand, and provide the scalability in a time of urgency?
All the information in the world is useless unless it can be accessed to provide timely insights, say, during a critical event. While many vendors provide face recognition as a standalone service, NEC’s face recognition solutions are part of a globally-trusted portfolio of biometric and safer cities solutions. From providing multi-modal forensics to cloud-based solutions, NEC backs up its face recognition solutions with tried and tested capabilities in related areas.

4. Is the technology open or locked in for future upgrades?
This is key to organisations looking to build a system that can be upgraded in the years ahead, as face recognition technology is improving all the time. NEC’s system is flexible in the way it works with other installations. It also plugs right into our safer cities solutions, which optimise results in face recognition by making use of its findings in various related scenarios.

Visit nec.com/safety for more information.