Why Covid may mean more facial recognition tech

Like many people who began to wear a face mask during the pandemic, Hassan Ugail quickly noticed one or two technical niggles.

His iPhone started having trouble recognising his face, which is how he preferred to unlock the device when out and about.

"I kind of have to take my mask off," says Prof Ugail, an expert in facial recognition at the University of Bradford. "I would rather it let me in by just looking at my eyes."

Coincidentally, research he conducted with one of his PhD students that was published last year had shown that half a face was enough for a specially trained facial recognition algorithm to work.

But out in the wild, some commercial systems that authenticate people via their faces were now stuttering thanks to the rise of masks.

The problem was highlighted in a July report from the National Institute of Standards and Technology (NIST) in the US. Researchers found the error rates of 89 different facial recognition systems they tested increased when the mouth and nose or bottom half of an individual's face was obscured - in some cases from less than 1% to as much as 50%.

Firms have raced to update their algorithms so that they can cope with masks, especially in situations where facial recognition is used to confirm someone's identity - such as authorising access to a secure building.

An updated report from NIST has now found that several algorithms, reconfigured since the pandemic began, make far fewer mistakes when analysing masked faces. In some cases, error rates were ten times better than before.

"Developers are indeed adapting their algorithms to handle face masks." says Mei Ngan, a computer scientist at NIST.

In their tests, Ms Ngan and her colleagues used a variety of superimposed shapes to represent different sizes of mask, not photos of mask-wearers.

The progress means facial recognition could now become even more widespread than before with companies marketing it as a contactless, potentially more hygienic, means of verifying identities in public places.

More Technology of Business

Prof Ugail has been tinkering with his own systems, used for research, so that they can continue to identify individuals even when restricted to analysing the area of the face around the eyes. The results of his experiments have not yet been published or peer-reviewed but he says the findings are promising, suggesting accuracy of around 86%.

One firm that says its algorithms have continued to cope well during the pandemic is Swiss-based Tech5. Although the NIST report found that masks caused the error rate of firm's system to jump from 0.45% to up to 14%, co-founder Rahul Parthe says the algorithm remained reliable enough in real-life settings.

"If somebody was wearing a mask and sunglasses and had a hat then you were basically missing a lot of their faces," he explains. "That's where we had to put in some requirements where a person would have to remove their hat or mask."

Tech5's systems are used in factories and schools in South East Asia, adds Mr Parthe, where they monitor daily attendance, for example.

And in schools, the system can perform an anonymous headcount, to see how many pupils were present on a particular day.

In these cases, where Tech5's algorithm is passively monitoring images from a CCTV camera in a corridor, for instance, Mr Parthe says mask-wearers are asked to avoid confusing the system by not wearing hats as well.

However, he argues that his firm was well-positioned to adapt to the rise of face masks because, having so many clients in Asia, Tech5's database had already been trained using lots of images of people wearing masks or religious garments that obscure part of the face.

"We never fine-tuned our algorithms just for face recognition with masks [during the pandemic]," a company spokeswoman says.

Controlling for masks has, until now, been less important in the West, notes Shaun Moore, founder and chief executive at Trueface: "The onset of Covid had really forced companies like Trueface to rethink what areas of the face are more important for facial recognition."

France-based Thales, which supplies facial recognition systems to airports and businesses for controlling access to secure areas, says it is able to obtain 99% accuracy for matching someone to a high quality reference image such as a passport photo - but only now that it has completed some additional training of its algorithm to focus on the eye area of faces.

Trueface took a similar approach, says Mr Moore. His staff began tweaking the firm's systems as early as February, when the spread of Covid began to become serious outside China.

But far from being a hiccup, Mr Moore argues the pandemic will ultimately accelerate the rollout of facial recognition, particularly for authentication purposes.

In October, his firm announced it had won a contract to install facial recognition access control systems at two US Air Force bases - a move designed specifically to reduce contact between people at the bases, for example, via touchpads and surfaces. Mr Moore thinks venues open to the public, including sports stadiums, will also turn to "contactless" identify verification.

"Facial recognition becomes the way to do that without having to hand over a key card, badge or ticket, it's reducing the contact you would have as the visitor, employee or spectator," he says.

"As that worker, you don't want to be touching a thousand cards."

It's important to note that, while general hygiene and hand-washing remain important in the pandemic, the US Centers for Disease Control and Prevention has said that although Covid-19 spreads easily from person to person, it is less commonly transmitted via contact with a contaminated surface.

Nonetheless, Covid-19 is clearly not only creating challenges for the proponents of facial recognition - but potential opportunities as well.