Expanding access or compounding inequality? Gender, finance, and AI

In Silicon Valley, smart fintech – financial technology that utilizes AI – is touted as an objective means by which to expand global access to loans. But is this new technology really objective? At a recent Faculty Research Fellows talk at the Clayman Institute for Gender Research, Postdoctoral Fellow Genevieve Smith argued that such technology is "a social and political technology that can reproduce and legitimize gender inequality under the guise of neutrality."

Historically, the concept of credit scoring, as Smith explained, originated in the United States, which subsequently exported the concept, its rules, and infrastructure globally. Within the realm of smart fintech, Smith focuses on smartphone applications that utilize machine learning (ML) to assess a person's creditworthiness for loans in low- and middle-income countries. To assess creditworthiness, ML-based lending apps aggregate a wide range of data, including alternative data accessible through a person's smartphone (e.g. SMS and social networks), to algorithmically determine an individual's "ability to pay" and "willingness to pay." 

Smith conducted qualitative interviews with managers, data scientists, and investors of largely U.S.-based digital lending fintechs developing AI-based credit scoring apps. These apps are understood as offering free and open access to loans for communities who have historically been unbanked or underbanked. In relying on machine learning, according to company interviewees, the digital lending app provides a fair and truthful assessment of a person's creditworthiness. Smith’s research, however, reveals the structurally gendered nature of smart fintech algorithms. Because of structural inequalities, women globally are more often active in the informal economy, more likely to work part time, and often have variable incomes because of time spent performing unpaid care. When these variables inform the fintech algorithms, the result is a bias in favor of men vis-à-vis creditworthiness.

At the same time, Smith's research uncovered a fundamental tension. While men receive larger loans and experience higher rates of approval, fintechs assert that women are better repayers. That is, in making late payments – and therefore being inferior repayers – men are more likely to pay late fees, thus becoming profit centers for the fintech companies. Despite being better repayers, women are approved for fewer loans and smaller amounts. As Smith summarized: "Creditworthiness as male is embedded under profit priorities while defended as ethically sound." 

Smith also analyzed a dataset of more than 200,000 Google Play user reviews to assess how borrowers described their own experiences using the app. For individual users of the app, the technology was described as largely positive. One woman from among Smith's dataset described the app as providing security and hope. Due to differences in digital access, and to lower levels of digital and financial literacy, however, women are less likely overall to use fintech apps.

Despite altruistic beliefs and positive user experiences, Smith's research uncovers persistent gender differences in financial access. “Technology shapes gender norms and relations: a system that systematically deems women less creditworthy doesn't just reflect existing inequality, it legitimizes and deepens norms around who is the breadwinner and who is the caretaker."

According to Smith, the stakes of gendered algorithms extend beyond algorithmic credit scoring. "When AI systems claim neutrality, they become harder to challenge,” she said. “It gets harder still when these tools are also framed as tools of social good.” Understanding claims of neutrality and their accuracy is essential for both financial inclusion and for any ways AI interacts with inequality.