Gender Bias In, Gender Gias Out: Providing Businesses with the Right Solutions to Mitigate Gender Bias in AI

It is striking how AI is infiltrating deeper and deeper into people’s lives, often in unexpected ways that challenge society’s collective assumptions. Today, AI aids in the detection of tumors and cancer, the discovery of new medicines and the execution of surgery in the medical field. Netflix, YouTube and Spotify’s AI algorithms monitor online content consumption and recommend movies, videos and songs to users. Many AI technologies are so well embedded into everyday life that people may be unaware of their use.

As AI continues to permeate all aspects of society, a number of ethical concerns must be addressed. Indeed, AI technologies have the potential to cause substantial harm if appropriate safeguards are not put in place. In fact, studies have revealed gender scandals in AI that disadvantage women, such as Amazon’s allegedly anti-women recruiting tool or Goldman Sachs’ Applecard, where customers complained that women with the same credit standing were given lower credit limits than men with the same credit standing. These are not rare occurrences; a study conducted by Gartner predicts that by 2022, 85% of AI projects will yield incorrect results due to bias in data, algorithms, or the teams in charge of managing them.

Heated controversies of gender bias in AI, emerging mainly from the US were found in a wide range of areas such as: word embeddings, online advertising placements, online news, web search, credit scoring, facial recognition or even recruitment. The most recent case of gender-biased AI revealed by an independent research audit is that of Facebook’s ad delivery algorithm, which showed different job advertisements to men and women despite the positions requiring the same qualifications. This indicates that, even when advertisers target a gender-balanced audience, Facebook’s job ad distribution is gender biased. The researchers confirmed that Facebook violated anti-discrimination laws, as its ad delivery can result in a skew in job ad delivery by gender that exceeds what can be legally explained by potential qualifications discrepancies.

As it became clear that AI technologies should not be entrusted with human decisions, a number of groups from industry, civil society, research and government organizations began developing mitigation tools and solutions. These experts urge for greater accountability in the design and usage of algorithms. This has contributed to a greater understanding of AI’s negative side-effects and provides a foundation for what fair AI should look like. A starting point may be to define fairness metrics to expose gender bias in issues such as allocation, quality of service and representation.

A burgeoning research paradigm of fair AI, which aims to detect and reduce unfairness in AI algorithms, has emerged in response to this issue. It has focused on how to define fairness, how to develop fairer algorithms and how to benchmark gender bias in a variety of settings as well as concepts derived from those definitions. Many papers have been written proposing fairness definitions and then generating best approximations or fairness guarantees based on hard constraints or fairness metrics using those definitions. One can  argue that there will be no simple or universal solutions because gender bias in AI is layered in techniques that are complex to define, resolve and manage.

At least twenty-one mathematical concepts of fairness were identified, each with their own variations and implications for reducing gender bias. Among these concepts is ‘equality of opportunity’ which considers whether each category is fairly likely to predict a desired outcome based on its real base rates; or also ‘disparate mistreatment’ a corollary that takes into account variations in false positive rates between groups. Another fairness meaning entails considering counterfactual situations in which members of protected groups become members of the non-protected community instead. This assumes that the system is unequal to the degree that results differ; a woman classified by a fair system should be classified the same way she would have been classified in a counterfactual situation in which she had been born a male. Finally, once a model has been developed, it can be used in a variety of ways, which may raise additional fairness concerns. The extent to which a model may have different effects on different groups may not be apparent until it is implemented in a decision-making system. Nonetheless, three formal and more widely accepted theories of fairness have emerged in recent years: 1) anti-classification, in which protected features such as gender, ethnicity and their proxies are not specifically used to make decisions; 2) classification parity, in which common measures of predictive success (e.g., false positive and false negative rates) are equivalent across classes identified by the protected attributes; and 3) calibration, in which outcomes are independent of protected attributes, conditional on risk estimates.

More conservative AI experts argue that if society hasn’t reconciled these gaps, AI systems can’t be expected to and that any solution will have trade-offs. Gender bias and fair AI are sociotechnical problems, meaning that AI systems can be unfair for a number of reasons, some of which are social, some of which are technological and some of which are a mixture of the two. They hereby propose achieving fair-AI by embracing a socio-technical approach.

There are significant technical efforts underway to identify and eliminate bias in AI systems, which are taking shape through communities such as Fairness, Accountability and Transparency in ML (FATML).

Open-source fairness toolkits have been developed in recent years to assist AI practitioners in assessing and mitigating unfairness in the systems they build. These are collaborative online platforms that enable various organizations to record, share and access contextual and experiential knowledge in order to promote fairness in AI systems. FairLearn by Microsoft as well as IBM’s AIFairness360 are examples of such toolkits. Practitioners can use these toolkits to assess the efficiency and actions of their AI systems against a variety of (un)fairness measures. Some toolkits also allow practitioners to apply algorithmic methods to their AI models to improve their fairness metrics.

Public and private organizations have published a near overwhelming collection of ethical principles and guidelines. Private organizations, such as Accenture, SAP, EY and Deutsche Telekom have published high-level corporate AI guidelines to steer the ethical development and deployment of fair AI systems. Those include SAP’s guiding principles for Artificial Intelligence, PwC responsible AI framework, Deutsche Telekom AI Guidelines, Sony Group AI Ethics Guidelines amongst others. In terms of public high-level principles, notable examples include the OECD’s AI Principles (2019) and the European Union Ethics Guideline for Trustworthy AI (2019), both of which can be used to shape public policy.

Another mitigation solution which has been acknowledged in many ways is the audit of AI algorithms whereby an independent party may evaluate the system for gender bias and other unintended consequences. In ‘Gender Shades’, researchers audited commercial facial recognition to assess their performance at classifying faces by binary gender and to determine if there were accuracy disparities based on gender or race. They specifically evaluated Microsoft, IBM and Face++’s commercial gender classification systems, which revealed that darker-skinned females are the most misclassified group, with error rates of up to thirty-five percent, whereas lighter-skinned males have a maximum error rate of less than one percent. While it is obvious why audits could aid in the development of public trust, developing such a mechanism is difficult and how they are to be carried out remains an open question and an area of active research.

By developing interdependencies among the bias mitigation solutions developed, companies can transition from bias mitigation as a simple add-on to bias mitigation as a strategy that is integrated throughout its entire lifecycle. Creating interdependencies between solutions has the advantage of placing an equal emphasis on technical and non-technical solutions, in contrast to current practices which have approached gender bias in AI primarily as a technical problem to be solved.

Given that the majority of the bias mitigation strategies proposed in the findings involve the active participation of both humans and machines, AI systems should be developed with the selective inclusion of human participation, rather than removing human involvement from a task. In essence, it implies that, rather than viewing gender as a bias issue to be addressed, all AI requirements should take gender into account holistically. It supports Meredith Broussard’s view of techno-chauvinism, which states that non-technical solutions are to be offered in addition to technical ones to approach gender bias in AI.

Companies that fail to implement organizational change and AI ethical implementations risk jeopardizing their bottom line in the long run. Currently, projects aimed at mitigating gender bias in AI in businesses are taking shape, thanks to internal advocates who take on additional responsibilities to ensure the company addresses ethical issues related to its AI system. This task cannot be left solely in the hands of a few but must be driven by the entire organization. As such, corporate initiatives namely incentives, standards, and feedback loops with regards to gender bias mitigation are to be implemented throughout the entire organization to build a systematic approach to mitigating gender bias in AI. Internal advocacy and change management would be full-time jobs assigned to people with the necessary skills, training and motivation. Individuals working on ethical AI would then be able to focus on their specific job rather than changing the job environment in order to do their job. In the aspirational future, corporate structures and processes would fully provide mechanisms for monitoring and adapting system-level practices to incorporate and address emergent ethical concerns, allowing individuals concerned about bias mitigation to devote their time and labor to making progress within their functions. Such changes should not be viewed as linear shifts from one fixed state to another, but rather as persistent measures and coalition building within the ever-changing nature of organizational contexts themselves.

There is an pressing need to strengthen the dialogue among stakeholders from various disciplines and backgrounds can assist companies in addressing gender bias in their AI systems. Leading thinkers in the emerging field of AI bias are overwhelmingly female, such as Joy Buolamwini, Timnit Gebru, Meredith Whittacker, Meredith Broussard, Margaret Mitchell, Sarah Myers-West, Kate Crawford and so on. This implies that those who are potentially affected by bias are more likely to recognize, comprehend and attempt to resolve it. This stresses the importance of bridging the gap between AI experts, non-AI experts, AI practioners, AI users and moreover all decision makers. One method is to make AI systems a cross-responsibility across the organization. As the design and evaluation of AI systems is rooted in different perspectives, concerns and goals, solutions to gender bias in AI necessitate robust interdisciplinary collaborations that incorporate technical, legal, social and ethical dimensions.

Building communication channels throughout the company to create opportunities for different groups to exchange perspectives is one way companies can promote multi-disciplinary and diverse conversations. Hackathons are one example to draw multi-disciplinary and diverse stakeholders into one room to brainstorm gender bias mitigation in AI. Developers, business leaders, policymakers and citizens in general should be encouraged to participate in certain types of conversations. The direct result of involving diverse stakeholders with diverse perspectives is to raise awareness about the risks and benefits of AI, allowing them to be better prepared to ask the tough questions and manage the right risks. Previous research has drawn attention to the importance of collaboration between engineers and domain experts who are knowledgeable about historical inequalities, cultural and social areas of concern for future AI development. Other perspectives in research had also been used, by explaining how incorporating decades of research on how gender ideology is embedded in language is a first step toward preventing the generation of biased algorithms.

Upcoming discussions centered on gender bias in AI will have to deal with an increase in social and technical complexity, as well as perspectives from various professions. This could result in a new and diverse AI ethics discipline that influences various existing professions such as philosophy, psychology, law, software and data.

As AI systems become more widely used, there is a growing emphasis on protecting people from harm while also equitably sharing the benefits of these systems. The impact of AI technologies on people cannot be altered without taking into account the people who create them as well as the organizational structure and culture of the human systems under which they function. Indeed, navigating these questions in organizations with prevalent practices necessitates skills that are not always part of the regular conversation within the workplace. Investing in providing practitioners with the resources and organizational insights they need to face the challenge of mitigating gender bias in AI is a crucial challenge that needs to be addressed correctly. The anticipation is that this will assist society at large in moving toward inclusive AI practices in other organizational settings, which connects different actors, resources and organizations to solve this multifaceted problem in similarly complex environments.

Above all, building greater trust and democratic participation in AI systems necessitates better organizational frameworks, Human-AI interaction methods, socio-technical platforms, tools and public engagement to increase critical public understanding and agency.