Facial recognition is all around us. Chances are you unlocked your phone using your face when you any notification on your phone. Facial recognition has crept into our lives very quickly and with little warning. And why wouldn’t it – with all that it offers! It can connect us with long-lost friends, help us find a lost child or a disoriented elderly, save us from losing money to fraud and so much more.
So, what is facial recognition and how does it work? In simple words, facial recognition is the process of using software to match 2 images of faces. However, the phrase “facial recognition” is often generically used to refer to one of 3 types of facial recognition algorithms – detection, analysis, and identification. The underlying principle behind these algorithms is the same – we have an image on an X and Y plane and we try to teach a machine how to find the x and y coordinates of whatever interests us in that image. In our case, it’s a face! A machine could return only the x and y coordinates of where a face is in the image. This is face detection. A machine could return the x and y coordinates of the eyes, the nose, and the lips of a face in the image, or in a more advanced solution, return the x and y coordinates of all the points that outline the lips. This is facial analysis. And finally, a machine could run the coordinates of a face’s features against coordinates of billions of faces to find a match. This is facial identification.
We serve unlimited knowledge with zero sign-up fees. Please enter your email address to subscribe and receive notifications of new articles by email. Remember, knowledge is the new currency that serves the best return!!!
As you can tell, facial recognition can have many different applications depending on what the machine was trained to do. Face detection is not uncommon in the world of photography where detected faces are used to adjust the lens’ focus making a human subject stand out in a photograph. This technology, on its own, is harmless. Since facial features are not being mapped into coordinates, it is nearly impossible to create a faceprint or identify a person from it.
Facial analysis, AKA face printing, may be used to understand emotions. It can tell us if the face in an image is happy or sad or worried or excited. Face printing has also had a recent rise in use on social media through features like Snapchat filters where coordinates of a user’s facial features are identified and then adjusted by the software to generate a different, often comical, version of the face. This is a much loved and used feature, however, it could be misused by a nefarious entity since users are willingly giving access to their face prints to the company.
Finally, the most common application of facial recognition is facial identification. If we trust our law enforcement agencies and governments to use this technology the right way, it is encouraging to know that they have been using facial recognition heavily, for a very long time, preventing as well as solving crime and more generally surveilling high-risk areas, like airports and other areas of public gathering. A version of facial recognition is also used by individuals, like you and me. That should more accurately be called facial verification. We use it on personal smartphones or even smart homes to verify that we are the owner, thus authenticating use. On a much larger, we use facial identification to tag friends in photos on a personal cloud, like Google Photos, or on a public cloud, like Facebook or Instagram. If not used responsibly, however, this technology can also be misused, especially if we are making our faceprints available publicly.
Our face is our unique identifier, like our signature or our fingerprint. We don’t leave those lying around in places they can be stolen from and misused. But we may be unintentionally leaving our faceprint all over the internet for it to be misused. With the rising concerns around the possible misuse of faceprints, policymakers are under a lot of pressure to come up with laws that balance the convenience of this very easily accessible biometric, our face, and the safety as well as privacy of an average citizen. But this technology is developing faster than policymakers can keep up with. So, in the meantime, it’s up to us to be informed and careful about how much of it we let into our lives.
Tech giants like Facebook have been penalized in recent years for what’s known as “Function Creep”, where data, including our faceprints, was collected for one purpose but used for another. After a series of similar penalties, Facebook announced the deletion of billions of faceprints they had collected over the years from photos uploaded and tagged by Facebook’s users. They added that until policymakers make the place for facial recognition in society clear, its use should be limited. Other big-data users with access to their users’ faceprints could soon follow suit. The technology is still far from being perfect – there have been numerous instances of misidentified individuals. However, the bigger risk is the amount of polarization this technology can create if scientists are not careful about what data they use to train their machines. A machine can only learn from examples and if the training data does not sufficiently represent minority groups, it could catalyze significant societal issues. Today, the biggest among them are racial profiling (e.g., considered a crime suspect primarily based on skin color) and creating gender norms (e.g., considered a man or a woman primarily based on adorned hairstyle).
Despite the seemingly big concerns around the use of unregulated facial recognition, however, the benefits seem to far outweigh the risks, especially in a few industries like Retail. Research and innovation in the field have grown this fast and successfully in no other industry. Several stores in the United States like Macy’s and Apple Stores use this technology for security reasons, including fraud prevention. It helps them, for example, to identify a previously convicted shoplifter and keep an eye on him while he walks through the store. Amazon Fresh stores are piloting the use of a similar technology that allows shoppers to “Just Walk Out” with their shopping and see the cost charged on the credit card associated with their online Amazon accounts. China’s leaders in e-commerce, Alibaba, and JD are also on a journey to redefine retail and remove the barrier between online and offline shopping with their grocery stores. In India, companies like FaceX are developing and testing the use of facial recognition to personalize shoppers’ experiences when they are in a physical store, similar to what they would experience in an online store.Let’s take a deeper look at some of these use cases and how they are being developed and adopted by users:
Cashless payment: The adoption of contactless payment systems, like Google Pay and Paytm had been on the rise pre-pandemic but in the last couple of years, contactless payment seems to have become the default mode in most transactions, especially when purchases are made in-store. Payment through facial recognition takes this a step further in terms of convenience. It allows payers to make a payment without any personal physical device at all, not even a cell phone. This makes the entire process more seamless and much faster than when we scan a QR code or type out the seller’s username or phone number and enter our card details or even select one from a list of pre-saved credit cards to make a payment. With facial recognition, when the buyer is ready to pay, his face is scanned. This is used to find his online account and the associated preferred credit card details. This credit card is then automatically charged for his purchases.
Several facial recognition systems, named “Smile to Pay”, “Pay by Face” and “Just Walk Out” have been launched around the globe and are being used by millions of people. This technology has mostly received positive feedback given the ease of use and with the knowledge that improvements are continuously being made to the systems. Some of the concerns raised are around the need for more education and transparency about how the system works. Users say that would prefer to go through a registration process where they set up their face to be used for payment. Similar to how they would set up their Face ID on a new phone. Most current systems, however, use existing photos from users’ Government records like KYC forms and Driver’s licenses which may be looked upon as a breach of trust. The counter-reasoning shared is that the registration process can be long and therefore discourage users from adopting facial recognition despite it being a one-time process. And since they already have access to users’ photos and credit card details, why not use them! Ultimately, the trick would be to find the right balance between convenience and privacy, between adoption and trust.
Store surveillance and navigation: Facial recognition has been used in-store surveillance for a long time. It is primarily used to identify previously convicted shop-lifters and keep an eye on them. Facial recognition systems are installed in parts of the store where items were kept under lock and key pre-pandemic, like the jewelry section of a store. This enables a single employee to monitor all the high-value store areas as opposed to having multiple employees posted at different high-value areas to unlock an item when a customer wants to see it or purchase it. Facial recognition is also used to monitor traffic in different areas of a store to dispatch help only where it’s most needed given the scarcity of employees who are willing to physically come into work.
Personalized in-store experiences: Personalization is probably the most liked use case of facial recognition among consumers. This is because consumers can draw parallels from their positive online shopping experiences from product recommendations to personalized deals and advertisements. But with the additional benefit of being able to see an item in person before paying for it.
Personalization programs related to advertisement are being adopted aggressively in many brick and mortar stores. This identifies a face, much less personally and only at a demographic level to display an advertisement more catered to the person’s demography. These stores have screens with advertisements that dynamically change to be more appropriate for the person walking towards them. On the surface, this may seem intrusive with no benefit for the customer but think about all the times, you were able to snag those sneakers on a great deal or that cheap last seat on a flight because an advertisement for it popped up on your social media. We can expect similar benefits from in-person targeted advertisements as well.
Pilot programs for personalizing in-store experiences using facial recognition have also been launched by several retail stores, most of whom have started with a select few of their loyal customers whom they are unlikely to lose, even in the case of a negative outcome. These programs identify a customer and predict what products he might be interested to buy based on his combined online and previous in-store purchasing as well as browsing history. An instant push notification is then sent to the customer regarding discounts on those products, encouraging the customer to make the purchase and potentially saving him some money in the process if he was otherwise going to buy it at full price. So, don’t be surprised if the next time you walk into a store, you are greeted by your name, get a notification about an offer on the dress you have been staring at for a minute, contemplating if you need it, or not asked for your credit card at the time of checkout. The entire premise for using facial recognition for in-store personalization is that it will provide the same win-win situation that online personalization does for both customers and stores. And I believe that keeping an open mind about it and adopting it with knowledge and caution can advance how we make our regular purchases by leaps and bound.
