10
Integrating Human Rights Principles into Systems and Product Design

The symposium’s final panel session explored how human rights concepts such as participation, nondiscrimination, privacy, and accessibility can be considered and applied throughout the process of designing new systems and products and how doing so from the start makes for better systems and products. Alice Agogino, Roscoe and Elizabeth Hughes Professor of Mechanical Engineering, Emeritus, Graduate School Professor at the University of California, Berkeley, discussed the factors involved in human-centered, user-centric design and where consideration of human rights fits into that process. Wendy Hui Kyong Chun, Canada 150 Research Chair in New Media, Professor in the School of Communication, and Director of the Digital Democracies Institute at Simon Fraser University, examined how engineering systems, particularly those connected to digital spaces, relate to upholding democracy and human rights. Tamara Brown, retired Vice President for Sustainability at Linde, discussed the ways that participation and rights can be prioritized during the design process, including with regard to disruptive technologies and those that combat climate change. Katie Shay, Associate General Counsel and Director of Human Rights at Cisco, talked about how Cisco considers human rights as part of its product design. Following the presentations, Lindsey Andersen, Associate Director for Human Rights at BSR (Business for Social Responsibility), moderated a discussion among the panelists and a question-and-answer session with the symposium participants.

In introductory remarks, Andersen explained that BSR works with businesses to integrate human rights thinking and principles into the process by which they develop, design, deploy, sell, and use technology. As part of its services, BSR conducts human rights assessments of technology products to identify human rights risks and then helps companies determine how to address them. “Something that comes up almost all the time in that work is how choices related to product design really impact the human rights risks that exist for a product, whether that is what the user interface looks like, what kinds of data are collected or used by a system, choices about system architecture, choices about what kinds of information is shared with users, and so on,” said Andersen.

For example, artificial intelligence (AI) systems can pose a risk to privacy related to the data used to train or update an AI model. AI models may also pose discrimination risks associated with outputs that perpetuate historical biases in their training data. These risks, said Andersen, arise directly from product and system design choices, which means that mitigating those risks requires action throughout the design process.

DEVELOPMENT AS FREEDOM

Alice Agogino spoke about a program that she and her colleagues established based on the concept of development as freedom and a human right. Citing Nobel Laureate Amartya Sen, Agogino said that it is important that development places human rights and freedom at the forefront because everyone has the human right to shape their own destiny and not just receive the destiny given to them.

Design as freedom, said Agogino, is a human-centered approach that puts greater responsibility on the designer to understand what kind of life users have reason to value before developing solutions and to empower users through co-design of those solutions (Perez, 2019). This, she said, is the essence of “development engineering,” a transdisciplinary field combining engineering, design, the physical and biological sciences, and energy and resource development with social sciences such as economics, entrepreneurship, business, and public policy. Development engineering aims to create scalable technological interventions in accordance with the needs and wants of individuals living in complex, resource-limited environments.³³

Agogino explained that students in developmental engineering learn to design and implement technology solutions for low-income regions by mastering problem-solving, understanding political and cultural contexts, applying evaluation methods, and developing community-focused professional skills such as teamwork, communication, cross-cultural awareness, capacity building, and sustainable design. “What is really critical about development engineering is that it is based on an excellency across different disciplines and a tech savvy coordination with teamwork,” said Agogino. “What is different about it is that we are always interested in being driven by development constraints and goals and understanding those, but also always thinking ahead to the future on how we can scale our development project.”

The concept of design being a human right and empowering people to co-design their own community means questioning the norms that team members that might have when joining a project. Considerations might include, for example, the role of gender in how team members frame a problem, including whether the problem is framed in terms of breaking ground in gender equity or simply reproducing norms that already exist. In this case, team members might be asked at the start of the design process to consider how the product would be used differently by women and men, and to explore whether local roles or customs could be leveraged to empower women in ways that are meaningful and acceptable within the community.

As another example, Agogino discussed a project involving the Pinoleville Pomo Nation, a tribe in Northern California, which has experienced a very high unemployment rate with most members living in poverty. When the tribe approached her, the cost of energy had recently skyrocketed, and it was becoming too costly to heat and cool the poorly insulated trailers provided by the U.S. Department of Housing and Urban Development. A typical approach to designing a solution for this issue might have resulted in a proposal to better insulate the trailers and a reference to tribal culture. Instead, said Agogino, her doctoral student endeavored to understand the tribe’s long-term aspirations and goals and framed the project in the context of their goals of sovereignty, economic independence, and environmental harmony. The end result was culturally sensitive sustainable buildings and energy and water systems that incorporated native plants and were co-designed and co-built by tribal members (Shelby et al., 2012).

Agogino concluded her remarks by listing the six Rs of Indigenous research (Grant et al., 2022):

1. Respect: For Indigenous cultural identity, communities, and topics
2. Relationships: Connects to the concept of all of our relations and building relations with communities
3. Responsibility: Responsibility to share only when allowed, and to publish ethically while considering ownerships, protocols, and community practices
4. Reverence: Respect for the sacred
5. Relevance: Legitimize and incorporate Indigenous knowledges into curriculum when it makes sense
6. Reciprocity: Both receiving and giving with communities

Agogino stated through her experience with Indigenous research methods, she has realized the need to consider the possible effects of one’s research seven generations into the future.

FINDING PLACE: INTERSECTIONS OF HUMAN RIGHTS AND ENGINEERING

Tamara Brown briefly discussed the evolution of the design process since she was a student, when human factors or human rights were only anecdotally mentioned in the design process, to now, when students learn about biomimicry³⁴ and Indigenous and traditional medicine as factors to consider in the process. In the clean energy space, she noted, there is now an emphasis on equity and community participation and benefit as was exemplified by the Biden administration’s Justice40 Initiative.³⁵

Brown added that human rights do not represent an addition to the work of engineers, because solving the right problem the right way is why engineers come to the table in the first place, and “you can’t do that without considering human rights.” She emphasized that human rights are relevant across all design phases and the product/application lifespan.

In the early days of her career, when she was in the medical device field, Brown learned about the importance of considering a representative population during the beta testing phase and having a design team that assesses relevant health disparities, as well as the value of working with external groups that can help inform problems and solutions. Using clean energy as an example, she noted that it is vital to understand people’s perceptions of new products, including their safety, and to consider how people are able to participate in the economies created around them. “If we want to develop applications that are very successful, we cannot develop those applications and leave communities behind,” said Brown. Communities, she added, must be able to ask questions about technologies, have the freedom to inform the development of those technologies, and have access and training to comfortably participate in those technologies.

Brown highlighted one human rights challenge for engineers: the need to develop alternatives to materials widely used in everyday electronics but sourced from conflict regions and produced using child labor. In that regard, impactful, intentional design is a win-win-win because it increases societal resilience, leads to technical success, and produces sustainable

business success. The outcomes from impactful and intentional design thus benefit not only the people using an application but also everyone in the supply chain. As Albert Einstein said, “Concern for man and his fate must always form the chief interest of all technical endeavors. Never forget this in the midst of your diagrams and equations.”

CISCO’S RESPONSIBLE AI JOURNEY

Katie Shay said that she values the concept of bringing a human rights framework to the design process because it applies regardless of context or a specific problem. When Cisco began its work in 2018 to develop responsible AI, it developed a framework for the predictive AI environment that it has since found applies to a generative AI environment with “very positive results.” Although new challenges have arisen with generative AI, the human rights challenges are similar, and having this framework in place, grounded in human rights principles, has enhanced the company’s ability to respond to these challenges and to its customers and end users.

When she joined the company in 2018, Shay worked to develop a dedicated human rights program that would apply the United Nations (UN) Guiding Principles on Business and Human Rights across corporate decision making and functions. The company had a human rights policy dating back to 2012, which it developed with the help of Andersen’s organization. Prompted by a question from a customer about the company’s position on responsible AI, she and her team went further and assembled a cross-functional working group with representation from government affairs, security, privacy, communications, and human rights to design a responsible AI program.

Shay and her team worked with the company’s engineers to understand how they designed and built the company’s technologies and then to develop requirements specific to responsible AI development that incorporated a human rights perspective. The company also launched a governance program with executive sponsorship from the engineering department and representation across the company. Cisco published a set of six principles—transparency, accountability, security, fairness, privacy, and reliability—to share with its customers and other stakeholders.³⁶ The company also hired a team to oversee AI governance full-time, conduct assessments, and bring issues to the governance board for review.

The company translates these responsible AI principles into controls that its engineers can apply to model creation and training data selection. These controls, said Shay, embed security, privacy, and human rights by design throughout a model’s life cycle and its application in products, services, and enterprise operations. Applying privacy principles, for example, means minimizing the personal data that the company collects and how long it retains them, as well as having robust processes in place so that the company can follow a principled approach to responding to government requests for data that respects human rights.

Shay said that her team has also developed an AI incident management process based on its privacy and security incident processes. The process focuses on bias, and if an individual or group identifies a potential bias issue, they can go through a formal channel to inform the company.

As an example of Cisco’s application of its responsible AI framework, Shay discussed the virtual backgrounds in its Webex product for web and video collaboration. Before releasing the Webex virtual background feature, the company found that it did not work well for certain hair textures and hairstyles and for certain lighting conditions. Engineers determined the need for more differentiated data in the training dataset. One unexpected finding was that oversampling training data from women improved the overall efficacy of the feature for everyone.

SOCIAL MEDIA ALGORITHMS AND HOMOPHILY

Wendy Chun said it is not that engineering needs to add human values or transdisciplinary concerns to the design process, but rather that it needs to embed better human values and engage other disciplines in a more deliberate manner. She then defined homophily as the axiom that similarity breeds connection, or in colloquial terms, birds of a feather flock together. Homophily is also the love of the same. It is the principle that grounds recommendation systems, social media networks, nearest neighbor algorithms, and the word embeddings that make up large language models (LLMs). “It is through homophily that we are clustered into network neighborhoods,” said Chun.

An important feature of homophily and social media algorithms, she said, is that someone’s most “valuable” likes are those that are slightly controversial or unusual and they have an outsized effect on the algorithm. For example, liking a popular media figure is not very useful algorithmically, but liking an atypical one would be. Chun noted that early in the development of social media algorithms, the hate button was weighted more heavily than the like button. This means, she asserted, that social media echo chambers were not an unfortunate error but rather an outgrowth of a system that groups people with similar likes because shared hatred draws them together even as it agitates them.

The term homophily originated from studies of residential segregation. It was coined by sociologists Robert K. Merton and Karl Lazarsfeld, who studied two housing projects in the northeastern United States in the early 1950s—one white and the other biracial. The researchers concluded that homophily does not always apply universally. They found that although people often form connections based on shared values, relationships across differences (heterophily) also occur. However, Chun observed, the study is now regarded as flawed because its analyses excluded residents who held mixed views on segregation (so-called ambivalents)—the largest category of white residents—as well as Black residents.

Chun concluded by positing about what can happen if we work intentionally across disciplines to build models that engage the entire range of human experiences. She proposed starting with the gaps that are central to networks. By addressing these gaps—filled with ignored voices—more inclusive and effective systems can be developed.

DISCUSSION

To start the discussion, Andersen asked the panelists for tangible examples of circumstances when incorporating human rights into the product design process made products better for everyone. Agogino replied that a product might be good for one group and not for another, but the people involved in the co-design process benefit. Shay offered the example of AI-generated closed captioning, which improves accessibility but has wider benefits.

Andersen then asked the panelists for their thoughts on the view that designing for the most vulnerable populations results in a product that works well for everyone and how that approach to design might be accomplished in practice. Brown stressed the importance of first knowing the intended population and the problem it wants addressed from its perspective. Chun said that she is working with colleagues who are trying to build LLMs and natural language processing techniques for Indigenous language reclamation. What is key here, she said, is the need to attend to Indigenous language and data sovereignty. Researchers benefit from this insistence on data sovereignty, despite the interesting challenges it presents.

Agogino and Shay both stressed the importance of working from a human rights framework throughout the design process and when considering a product or application’s life cycle. Shay noted that the UN Guiding Principles on Business and Human Rights use a framework of scale, scope, and severity of harm that prioritizes risk assessment and mitigation as parts of the human rights assessment.

When asked to identify the largest constraint to integrating human rights into the design process, Brown said that there is a lack of awareness, but one she sees being addressed in today’s engineering design students who are more informed about social, economic, and environmental justice. She noted that, for companies, the challenge often relates to cost, and therefore the people leading sustainability efforts in corporate engineering positions should emphasize this challenge and devote time to addressing it. Brown offered an example regarding water and human rights, saying that chemical engineers need to consider water scarcity and whether their designs are respecting the scarcity of water and the human right to have access to water.

Shay remarked that most of her engineering colleagues do not have personal expertise or education in human rights, and incorporating human rights into engineering education, such as courses applying the UN Guiding Principles Framework in engineering schools, could be beneficial to the field. Meanwhile, she said that engineers have expressed an interest in working with her team at Cisco and that they have been receptive to trainings. She also noted that direct engagement with customer and communities that will use a technology solution can give engineers a deeper understanding of their perspectives and needs so that they can design more effectively for them.

Agogino noted that a key challenge is examining engineers’ own work environments to ensure they foster the openness necessary for considering human rights issues. Closing out the session, Chun observed that when she holds workshops with engineers, community members, and artists, engineers are excited and feel empowered when they are challenged to rethink their default assumptions.