WASHINGTON — Some uses of facial recognition technology raise significant concerns that merit a swift government response, says a new report from the National Academies of Sciences, Engineering, and Medicine. The report recommends consideration of federal legislation and an executive order, as well as attention from courts, the private sector, civil society organizations, and other organizations that work with facial recognition technology, and provides guidance for the technology’s responsible development and deployment. 

A powerful and increasingly used tool, facial recognition technology is useful for a large range of identity verification and identification applications, offering capabilities for checking whether someone is who they say they are and identifying a person in an image. Systems utilize trained artificial intelligence models to extract facial features and create a biometric template from an image, and compare the features in the template to the features of another image or set of images to produce a similarity score. The accuracy and speed of these systems have advanced rapidly in the last decade with the adoption of deep neural network-based machine learning, the report says.  

With few exceptions, the U.S. does not currently have authoritative guidance, regulations, or laws to adequately address issues related to facial recognition technology use, the report finds. It also notes that facial recognition technology can interfere with and substantially affect the values embodied in U.S. privacy, civil liberties, and human rights commitments — even if it does not necessarily violate rights and obligations included in statutes or constitutional provisions. 

“Facial recognition technology generates novel and complex legal challenges and raises a variety of distinct, unsettled legal questions. It also raises complicated social questions about privacy and public and private surveillance, given the highly personal implications of the technology,” said Jennifer Mnookin, co-chair of the committee that wrote the report and chancellor of the University of Wisconsin-Madison. “It is crucial that governments make tackling these issues a priority ― failing or choosing not to adopt policies and regulations on the development and use of facial recognition technology would effectively cede decision-making and rulemaking on these important questions of great public concern entirely to the private sector and the marketplace.” 

Areas of Concern

Facial recognition technology has become increasingly embedded into everyday life, with a wide spectrum of uses, the report says. Some of those uses are innocuous, such as allowing people to unlock their smartphones. But when applied broadly and without safeguards, the technology can allow repressive regimes to create detailed records of people’s movements and activities and block targeted individuals from participation in public life. Many uses and potential uses fall somewhere in between, creating a large gray area where individual assessments of risks, benefits, trade-offs, and values may vary, thus affecting how they should be regulated or if they should be permitted at all. The report recognizes the value of facial recognition technology and does not advocate for an overall ban, but it says that a number of uses may cause enough concern that they should be prohibited. 

“Facial recognition technology has the potential to impact civil liberties, human rights, and privacy in meaningful ways, because it changes the scale and cost of collecting detailed data about a person’s every move,” said committee co-chair Edward Felten, Robert E. Kahn Professor of Computer Science and Public Affairs emeritus and founding director of the Center for Information Technology Policy at Princeton University. “The number of uses will continue to expand as the technology becomes more widespread and inexpensive ― for example, it is likely only a matter of time before stores routinely scan customers’ faces upon entry to personalize shopping experiences and marketing, and perhaps more troubling, private individuals could potentially use it to target others.” 

The report says there are two principal categories of concern in facial recognition, though they can overlap. One is potential harms from problematic use or misuse of the technology, which become more salient as the technology becomes more accurate and capable. The second is potential harms from errors or limitations in the technology itself, such as when systems have different false positive or false negative rates for different demographic groups.  

The committee found that facial recognition technology intersects with equity and race in several key ways. Many systems deployed in the U.S. are trained using datasets that are imbalanced and disproportionately rely on data from White individuals. As a result, these systems have higher false positive match rates for racial minorities. They also provide law enforcement with a powerful new surveillance tool that can serve to reinforce patterns of or perceived need for elevated scrutiny, especially in marginalized communities, which may be compounded through law enforcement’s use of reference galleries based on mug shots. These issues create additional burdens for some groups of individuals, including African Americans and others that have been historically marginalized in the U.S. 

Some concerns about facial recognition technology can be addressed by improving the technology itself, while others require changes to procedures or training, restrictions on when or how facial recognition technology is used, and regulation of the conduct that facial recognition technology enables.  

Ensuring Responsible Use

There are multiple ways for the government to address issues raised by facial recognition technology. The report recommends that the Executive Office of the President consider issuing an executive order on the development of guidelines for the appropriate use of facial recognition technology by federal departments and agencies. Any executive order should also address both equity concerns and the protection of privacy and civil liberties. 

New federal legislation should also be considered to address equity, privacy, and civil liberty concerns; limit potential harms to individual rights by private and public actors; and protect against misuse of facial recognition technology. Specifically, the report says that legislators should consider: 

• Placing limitations on the storing of face images and templates. 
• Requiring training and certification of system operators and decision-makers, particularly for applications where errors can significantly harm subjects, such as in law enforcement. 
• Addressing specific use concerns, such as use of facial recognition technology for mass or individual surveillance, harassment or blackmail, access to housing, and other public and private uses that could intentionally or otherwise chill the exercise of political and civil liberties. 
• Passing a federal privacy law specific to facial recognition technology, or broader privacy legislation that deals with commercial practices compromising privacy. 

• Placing limitations on the storing of face images and templates. 
• Requiring training and certification of system operators and decision-makers, particularly for applications where errors can significantly harm subjects, such as in law enforcement. 
• Addressing specific use concerns, such as use of facial recognition technology for mass or individual surveillance, harassment or blackmail, access to housing, and other public and private uses that could intentionally or otherwise chill the exercise of political and civil liberties. 
• Passing a federal privacy law specific to facial recognition technology, or broader privacy legislation that deals with commercial practices compromising privacy. 

The federal government should work to establish an array of industrywide standards and methods with international and industry partners. These should include process standards in areas such as data security and standards defining minimum acceptable image quality, false positive and negative rates, and accuracy variation across phenotypes for given applications. The report recommends the National Institute of Standards and Technology as a hub for government activities.  

The report also recommends other avenues that the federal government should pursue to guide responsible use and develop of facial recognition technology, including: 

• The U.S. departments of Justice and Homeland Security should establish a working group to develop and periodically review standards for reasonable and equitable use by federal, state, and local law enforcement agencies. The group should work on issues relating to image quality standards; guidance on the use of facial recognition technology for real-time surveillance of public areas; transparency and community oversight; and mechanisms for redress for individuals harmed by system misuse or abuse. 
• Grants and other support for state and local law enforcement should require that recipients adhere to certain technical, procedural, and disclosure requirements related to the use of facial recognition technology. 
• NIST should sustain a vigorous program of facial recognition technology testing and evaluation to continue driving an increase in accuracy and a reduction in demographic differentials. 
• The government should also establish a program to develop and refine a risk management framework to help organizations identify and mitigate the risks of proposed facial recognition technology applications. 

• The U.S. departments of Justice and Homeland Security should establish a working group to develop and periodically review standards for reasonable and equitable use by federal, state, and local law enforcement agencies. The group should work on issues relating to image quality standards; guidance on the use of facial recognition technology for real-time surveillance of public areas; transparency and community oversight; and mechanisms for redress for individuals harmed by system misuse or abuse. 
• Grants and other support for state and local law enforcement should require that recipients adhere to certain technical, procedural, and disclosure requirements related to the use of facial recognition technology. 
• NIST should sustain a vigorous program of facial recognition technology testing and evaluation to continue driving an increase in accuracy and a reduction in demographic differentials. 
• The government should also establish a program to develop and refine a risk management framework to help organizations identify and mitigate the risks of proposed facial recognition technology applications. 

The study — undertaken by the Committee on Facial Recognition: Current Capabilities, Future Prospects, and Governance— was sponsored by the U.S. Department of Homeland Security and the Federal Bureau of Investigation. 

The National Academies of Sciences, Engineering, and Medicine are private, nonprofit institutions that provide independent, objective analysis and advice to the nation to solve complex problems and inform public policy decisions related to science, engineering, and medicine. They operate under an 1863 congressional charter to the National Academy of Sciences, signed by President Lincoln.  

Contact:

Josh Blatt, Media Relations Officer
Office of News and Public Information
202-334-2138; email news@nas.edu