Study Reveals Racial Bias in AI Decision-Making for Mortgage Approvals
A recent study by researchers at Lehigh University in Pennsylvania has uncovered significant racial biases in the recommendations made by chatbots used for mortgage application processes. The study highlights the discriminatory tendencies of these AI systems, casting a spotlight on an emerging issue within the financial sector's adoption of new technologies.
The research utilised a dataset of 6,000 sample loan applications based on the 2022 Home Mortgage Disclosure Act. The findings were stark, with the AI systems showing a tendency to deny loan applications from Black individuals more frequently than identical applications from white applicants. Additionally, the AI assigned higher interest rates to Black applicants and categorised Black and Hispanic borrowers as "riskier". Specifically, the study observed that white applicants were 8.5% more likely to receive loan approvals than Black applicants with identical financial backgrounds. The disparity widened in cases where applicants had a low credit score of 640, with white applicants being approved 95% of the time, compared to less than 80% for Black applicants.
This experiment aimed to examine how financial institutions utilise AI algorithms, machine learning, and large language models to expedite lending and underwriting processes. The findings have sparked concerns over the "black box" nature of these systems, where their inner workings remain opaque to users. This lack of transparency poses significant risks, as flawed data inputs and entrenched biases can lead to detrimental outcomes.
Donald Bowen, an assistant fintech professor at Lehigh University and one of the authors of the study, expressed concern over these results. He pointed out that while these AI systems can significantly lower operational costs in various industries, there is a substantial risk of perpetuating systemic biases embedded in historical data.
Machine learning algorithms employed in these AI systems typically function through classification models. These models use inputs such as age, income, education, and credit history to produce outcomes — in this context, whether to approve a loan. Advanced deep learning techniques enable these systems to process vast datasets, but if these data sets are biased, the outcomes will reflect those biases.
Michael Wellman, a computer science professor at the University of Michigan, noted that the problems arise when AI systems replicate historical disparities in decision-making, especially affecting groups that have been historically disadvantaged.
The Lehigh University study was motivated by earlier findings from a smaller class assignment, which revealed racial discrimination by chatbots. To conduct the official study, the team ran thousands of simulated loan applications through various commercial large language models, including OpenAI’s GPT 3.5 Turbo, GPT 4, Anthropic’s Claude 3 Sonnet and Opus, and Meta’s Llama 3-8B and 3-70B. Notably, when race was included as a variable, discrepancies were observed, whereas instructions to exclude bias resulted in no significant differences in outcomes.
The issue extends beyond race data — often not explicitly collected in lending — as other data points, influenced by systemic racism, carry their biases into AI decision-making processes. This includes factors such as credit scores, which can be affected by historical discrimination in the housing and labour markets, and zip codes, influenced by practices like redlining.
The implications of biased AI systems reach further into other domains, including hiring practices and judicial processes. Decision-making technologies have been adopted in various sectors, including employment, where AI platforms filter applications and pre-screen candidates. The use of such systems has elicited legal challenges, including a notable case involving the tutoring company iTutorGroup Inc., which settled a lawsuit after being accused of using AI to unlawfully discriminate against older applicants.
Judicial systems have incorporated AI for tasks such as risk assessment of defendants and determinations related to pretrial release. However, the potential for AI tools to exhibit bias raises concerns, prompting states like Utah to legislate for human oversight in AI-driven judicial decisions.
In response to these issues, legislatures across the United States have begun introducing measures to curb algorithmic discrimination. For instance, Utah and three other states have established laws to prevent AI systems from contributing to discriminatory practices based on race, gender, or other protected characteristics. Furthermore, pending bills in Congress seek to regulate AI's impact on financial systems, aiming to address biases and foster trustworthiness.
While companies are expected to continue integrating AI technologies due to their cost-saving benefits, researchers such as Bowen advocate for rigorous bias audits and increased human involvement in AI decision-making processes. As calls for industry standards and regulatory oversight grow, the challenge remains to ensure AI models are developed and deployed in a manner that promotes fairness and transparency.
Source: Noah Wire Services