Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models

Paper: arXiv 2510.05702

Authors: Fabrizio Dimino, Krati Saxena, Bhaskarjit Sarmah, Stefano Pasquali

Abstract

Large Language Models are increasingly adopted in financial applications to support investment workflows. However, prior studies have seldom examined how these models reflect biases related to firm size, sector, or financial characteristics, which can significantly impact decision-making. This paper addresses this gap by focusing on representation bias in open-source Qwen models. We propose a balanced round-robin prompting method over approximately 150 U.S. equities, applying constrained decoding and token-logit aggregation to derive firm-level confidence scores across financial contexts. Using statistical tests and variance analysis, we find that firm size and valuation consistently increase model confidence, while risk factors tend to decrease it. Confidence varies significantly across sectors, with the Technology sector showing the greatest variability. When models are prompted for specific financial categories, their confidence rankings best align with fundamental data, moderately with technical signals, and least with growth indicators. These results highlight representation bias in Qwen models and motivate sector-aware calibration and category-conditioned evaluation protocols for safe and fair financial LLM deployment.

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 7.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced statistical methods like ANOVA, Pearson/Spearman/Kendall correlations with FDR correction, and confidence interval analysis, indicating moderate to high mathematical complexity. It also demonstrates high empirical rigor through a systematic, multi-model experiment on 150 U.S. equities with rigorous data collection, multiple testing protocols, and statistical validation of findings.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Detect Representation Bias in LLMs for Finance"] --> B["Data & Methodology<br>~150 US Equities with Balanced Round-Robin Prompting"]
  B --> C["Computation<br>Constrained Decoding &<br>Token-Logit Aggregation"]
  C --> D["Outputs<br>Firm-Level Confidence Scores"]
  D --> E{"Statistical Analysis"}
  E --> F["Findings: Firm Size & Valuation ↑ Confidence"]
  E --> G["Findings: Risk Factors ↓ Confidence"]
  E --> H["Findings: Sector Variability<br>High in Tech"]

Cited References (46)

1. Large language models in finance: Opportunities and challenges (2023)
2. Predictably biased: The influence of position bias on language model predictions (2020)
3. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on Artificial Intelligence and amending certain Union Legislative Acts (Artificial Intelligence Act) (2024)
4. Large language models exhibit systematic position bias in multiple-choice questions (2023)
5. Calibrating factual knowledge in pretrained language models (2021)
6. Fairness in financial AI: Detecting and mitigating algorithmic bias (2023)
7. Judgment in managerial decision making (2013)
8. Global Sustainable Investment Review 2020 (2021)

3 of 46 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.