Representational Harm

Systems cause representational harm when they misrepresent a group of people in a negative manner. Representational harms include perpetuating harmful stereotypes about or minimizing the existence of a social group, such as a racial, ethnic, gender, or religious group. Machine learning algorithms often commit representational harm when they learn patterns from data that have algorithmic bias, and this has been shown to be the case with large language models. While preventing representational harm in models is essential to prevent harmful biases, researchers often lack precise definitions of representational harm and conflate it with allocative harm, an unequal distribution of resources among social groups, which is more widely studied and easier to measure. However, recognition of representational harms is growing and preventing them has become an active research area. Researchers have recently developed methods to effectively quantify representational harm in algorithms, making progress on preventing this harm in the future.

Types

Three prominent types of representational harm include stereotyping, denigration, and misrecognition. These subcategories present many dangers to individuals and groups.

Stereotypes are oversimplified and usually undesirable representations of a specific group of people, usually by race and gender. This often leads to the denial of educational, employment, housing, and other opportunities. For example, the model minority stereotype of Asian Americans as highly intelligent and good at mathematics can be damaging professionally and academically.

Representational harm happens when the representation of details teams improves damaging stereotypes, developing social exclusion and prejudice. This experience is particularly noticeable in the depiction of marginalised groups, containing people of color, women, LGBTQ+ people, and people with handicaps. Media depictions of these groups generally stop working to catch their array and intricacy. Instead, they are typically reduced to one-dimensional caricatures, which ultimately continue social prejudices. These organised depictions contribute to the help of hazardous stereotypes and the marginalisation of these locations.

Denigration is the action of unfairly criticizing individuals. This frequently happens when the demeaning of social groups occurs. For example, when searching for "Black-sounding" names versus "white-sounding" ones, some retrieval systems bolster the false perception of criminality by displaying ads for bail-bonding businesses. A system may shift the representation of a group to be of lower social status, often resulting in a disregard from society.

Research shows that hazardous depictions in the media can have substantial emotional and social impacts on both individuals and areas. Lawrence Bobo examined the issue of Ethnic stereotype in film, tv, and marketing. African Americans are commonly received duties specified by features such as "violent tendencies," "laziness," or being "merely for contentment features." While these representations might appear varied externally, they stay to boost underlying frameworks of white prominence and racial inequality. As a circumstances, Black individuals are frequently represented as law offenders or in secondary roles, which adds to the support of Ethnic stereotype and Institutional racism.

Misrecognition, or incorrect recognition, can display in many forms, including, but not limited to, erasing and alienating social groups, and denying people the right to self-identify. Erasing and alienating social groups involves the unequal visibility of certain social groups; specifically, systematic ineligibility in algorithmic systems perpetuates inequality by contributing to the underrepresentation of social groups. Not allowing people to self-identify is closely related as people's identities can be 'erased' or 'alienated' in these algorithms. Misrecognition causes more than surface-level harm to individuals: psychological harm, social isolation, and emotional insecurity can emerge from this subcategory of representational harm.

Quantification

As the dangers of representational harm have become better understood, some researchers have developed methods to measure representational harm in algorithms.

Modeling stereotyping is one way to identify representational harm. Representational stereotyping can be quantified by comparing the predicted outcomes for one social group with the ground-truth outcomes for that group observed in real data. For example, if individuals from group A achieve an outcome with a probability of 60%, stereotyping would be observed if it predicted individuals to achieve that outcome with a probability greater than 60%. The group modeled stereotyping in the context of classification, regression, and clustering problems, and developed a set of rules to quantitatively determine if the model predictions exhibit stereotyping in each of these cases.

Other attempts to measure representational harms have focused on applications of algorithms in specific domains such as image captioning, the act of an algorithm generating a short description of an image. In a study on image captioning, researchers measured five types of representational harm. To quantify stereotyping, they measured the number of incorrect words included in the model-generated image caption when compared to a gold-standard caption. They manually reviewed each of the incorrectly included words, determining whether the incorrect word reflected a stereotype associated with the image or whether it was an unrelated error, which allowed them to have a proxy measure of the amount of stereotyping occurring in this caption generation. These researchers also attempted to measure demeaning representational harm. To measure this, they analyzed the frequency with which humans in the image were mentioned in the generated caption. It was hypothesized that if the individuals were not mentioned in the caption, then this was a form of dehumanization.

Examples

One of the most notorious examples of representational harm was committed by Google in 2015 when an algorithm in Google Photos classified Black people as gorillas. Developers at Google said that the problem was caused because there were not enough faces of Black people in the training dataset for the algorithm to learn the difference between Black people and gorillas. Google issued an apology and fixed the issue by blocking its algorithms from classifying anything as a primate. In 2023, Google's photos algorithm was still blocked from identifying gorillas in photos.

Another prevalent example of representational harm is the possibility of stereotypes being encoded in word embeddings, which are trained using a wide range of text. These word embeddings are the representation of a word as an array of numbers in vector space, which allows an individual to calculate the relationships and similarities between words. However, recent studies have shown that these word embeddings may commonly encode harmful stereotypes, such as the common example that the phrase "computer programmer" is oftentimes more closely related to "man" than it is to "women" in vector space. This could be interpreted as a misrepresentation of computer programming as a profession that is better performed by men, which would be an example of representational harm.

Addressing representational harm

Initiatives to minimise representational harm include advertising for even more inclusive and accurate portrayals of marginalised teams in the media. Scholars and protestors recommend that the method to reducing representational injury depends on raising the selection of voices both behind and before the digital video camera. When marginalized groups are provided the chance to represent themselves, they can check traditional stereotypes and present their experiences additional authentically.

Over the last few years, efforts to increase representation of people of color, women, and LGBTQ+ people in conventional media have made some progression. Films such as Selma, routed by Ava DuVernay, and tv series like Pose, developed by Ryan Murphy, have actually been extensively applauded for their nuanced and respectful representations of marginalised communities. These tasks existing complex individualities and stories that move past streamlined stereotypes.

Self-representation is one more crucial method to addressing representational harm. By equipping marginalised locations to create their really own tales, media designers can effectively reduce the perpetuation of hazardous stereotypes. This procedure consists of both the manufacturing of media product by participants of these communities and proactively difficult typical media structures that have actually historically omitted them.

References

{{reflist

Wikipedia Student Program
Technology
Information ethics
AI safety