Fairness (machine Learning)

Fairness in machine learning refers to the various attempts at correcting algorithmic bias in automated decision processes based on machine learning models. Decisions made by computers after a machine-learning process may be considered unfair if they were based on variables considered sensitive. Examples of these kinds of variable include gender, ethnicity, sexual orientation, disability and more. As it is the case with many ethical concepts, definitions of fairness and bias are always controversial. In general, fairness and bias are considered relevant when the decision process impacts people's lives. In machine learning, the problem of algorithmic bias is well known and well studied. Outcomes may be skewed by a range of factors and thus might be considered unfair with respect to certain groups or individuals. An example would be the way social media sites deliver personalized news to consumers.

Context

Discussion about fairness in machine learning is a relatively recent topic. Since 2016 there has been a sharp increase in research into the topic. This increase could be partly accounted to an influential report by ProPublica that claimed that the COMPAS software, widely used in US courts to predict recidivism, was racially biased. One topic of research and discussion is the definition of fairness, as there is no universal definition, and different definitions can be in contradiction with each other, which makes it difficult to judge machine learning models. Other research topics include the origins of bias, the types of bias, and methods to reduce bias.

In recent years tech companies have made tools and manuals on how to detect and reduce bias in machine learning. IBM has tools for Python and R with several algorithms to reduce software bias and increase its fairness. Google has published guidlines and tools to study and combat bias in machine learning. Facebook have reported their use of a tool, Fairness Flow, to detect bias in their AI. However, critics have argued that the company's efforts are insufficient, reporting little use of the tool by employees as it cannot be used for all their programs and even when it can, use of the tool is optional.

Controversies

The use of algorithmic decision making in the legal system has been a notable area of use under scrutiny. In 2014, then U.S. Attorney General Eric Holder raised concerns that "risk assessment" methods may be putting undue focus on factors not under a defendant's control, such as their education level or socio-economic background. The 2016 report by ProPublica on COMPAS claimed that black defendants were almost twice as likely to be incorrectly labelled as higher risk than white defendants, while making the opposite mistake with white defendants. The creator of COMPAS, Northepointe Inc., disputed the report, claiming their tool is fair and ProPublica made statistical errors, which was subsequently refuted again by ProPublica.

Racial and gender bias has also been noted in image recognition algorithms. Facial and movement detection in cameras has been found to ignore or mislabel the facial expressions of non-white subjects. In 2015, the automatic tagging feature in both Flickr and Google Photos was found to label black people with tags such as "animal" and "gorilla". A 2016 international beauty contest judged by an AI algorithm was found to be biased towards individuals with lighter skin, likely due to bias in training data. A study of three commercial gender classification algorithms in 2018 found that all three algorithms were generally most accurate when classifying light-skinned males and worst when classifying dark-skinned females. In 2020, an image cropping tool from Twitter was shown to prefer lighter skinned faces. DALL-E, a machine learning Text-to-image model released in 2021, has been prone to create racist and sexist images that reinforce societal stereotypes, something that has been admitted by its creators.

Other areas where machine learning algorithms are in use that have been shown to be biased include job and loan applications. Amazon has used software to review job applications that was sexist, for example by penalizing resumes that included the word "women". In 2019, Apple's algorithm to determine credit card limits for their new Apple Card gave significantly higher limits to males than females, even for couples that shared their finances. Mortgage-approval algorithms in use in the U.S. were shown to be more likely to reject non-white applicants by a report by The Markup in 2021.

Group Fairness criteria

In classification problems, an algorithm learns a function to predict a discrete characteristic $Y$, the target variable, from known characteristics $X$. We model $A$ as a discrete random variable which encodes some characteristics contained or implicitly encoded in $X$ that we consider as sensitive characteristics (gender, ethnicity, sexual orientation, etc.). We finally denote by $R$ the prediction of the classifier.
Now let us define three main criteria to evaluate if a given classifier is fair, that is if its predictions are not influenced by some of these sensitive variables.Solon Barocas; Moritz Hardt; Arvind Narayanan
''Fairness and Machine Learning''
Retrieved 15 December 2019.

Independence

We say the random variables $(R,A)$ satisfy independence if the sensitive characteristics $A$ are statistically independent of the prediction $R$, and we write
$R \bot A.$
We can also express this notion with the following formula:
$P(R = r\ , \ A = a) = P(R = r\ , \ A = b) \quad \forall r \in R \quad \forall a,b \in A$
This means that the classification rate for each target classes is equal for people belonging to different groups with respect to sensitive characteristics $A$.

Yet another equivalent expression for independence can be given using the concept of mutual information between random variables, defined as
$I(X,Y) = H(X) + H(Y) - H(X,Y)$
In this formula, $H(X)$ is the entropy of the random variable $X$. Then $(R,A)$ satisfy independence if $I(R,A) = 0$.

A possible relaxation of the independence definition include introducing a positive slack $P(R = r\ , \ A = a) \geq P(R = r\ , \ A = b) - \epsilon \quad \forall r \in R \quad \forall a,b \in A$

Finally, another possible relaxation is to require $I(R,A) \leq \epsilon$.

Separation

We say the random variables $(R,A,Y)$ satisfy separation if the sensitive characteristics $A$ are statistically independent of the prediction $R$ given the target value $Y$, and we write
$R \bot A\ , \ Y.$
We can also express this notion with the following formula:
$P(R = r\ , \ Y = q, A = a) = P(R = r\ , \ Y = q, A = b) \quad \forall r \in R \quad q \in Y \quad \forall a,b \in A$
This means that all the dependence of the decision $R$ on the sensitive attribute $A$ must be justified by the actual dependence of the true target variable $Y$.

Another equivalent expression, in the case of a binary target rate, is that the true positive rate and the false positive rate are equal (and therefore the false negative rate and the true negative rate are equal) for every value of the sensitive characteristics:
$P(R = 1\ , \ Y = 1, A = a) = P(R = 1\ , \ Y = 1, A = b) \quad \forall a,b \in A$
$P(R = 1\ , \ Y = 0, A = a) = P(R = 1\ , \ Y = 0, A = b) \quad \forall a,b \in A$

A possible relaxation of the given definitions is to allow the value for the difference between rates to be a positive number lower than a given slack confusion matrix