Paper: arXiv 2311.15548

Abstract

The hallucination issue is recognized as a fundamental deficiency of large language models (LLMs), especially when applied to fields such as finance, education, and law. Despite the growing concerns, there has been a lack of empirical investigation. In this paper, we provide an empirical examination of LLMs’ hallucination behaviors in financial tasks. First, we empirically investigate LLM model’s ability of explaining financial concepts and terminologies. Second, we assess LLM models’ capacity of querying historical stock prices. Third, to alleviate the hallucination issue, we evaluate the efficacy of four practical methods, including few-shot learning, Decoding by Contrasting Layers (DoLa), the Retrieval Augmentation Generation (RAG) method and the prompt-based tool learning method for a function to generate a query command. Finally, our major finding is that off-the-shelf LLMs experience serious hallucination behaviors in financial tasks. Therefore, there is an urgent need to call for research efforts in mitigating LLMs’ hallucination.

Complexity vs Empirical Score

  • Math Complexity: 1.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper is primarily empirical, focusing on benchmarking LLM performance and evaluating mitigation techniques with datasets, API calls, and compute metrics, which requires high implementation and data effort. Mathematical complexity is low, as it lacks advanced theoretical derivations and focuses on experimental methodologies rather than dense formulas.

Research Flowchart

  flowchart TD
  Start(["Research Goal:<br>Examine Hallucination in LLMs for Finance"]) --> Input1["Financial Concepts &<br>Terminologies"]
  Start --> Input2["Historical Stock<br>Price Queries"]
  
  Input1 --> Method["Methodology:<br>Empirical Testing of LLM Capabilities"]
  Input2 --> Method
  
  Method --> Mitigate["Mitigation Strategies:<br>1. Few-Shot Learning<br>2. DoLa Decoding<br>3. RAG<br>4. Tool Learning"]
  
  Mitigate --> Finding(["Major Finding:<br>Off-the-shelf LLMs show<br>serious hallucination in finance<br>Urgent need for mitigation"])
  
  style Start fill:#e1f5fe,stroke:#01579b,stroke-width:2px
  style Finding fill:#ffebee,stroke:#c62828,stroke-width:2px

Cited References (50)

  1. Bloomberggpt: A large language model for finance (2023) — link
  2. Finbert: A pretrained language model for financial communications (2020) — link
  3. FinGPT : Democratizing Internet-scale Data for Financial Large Language Models (2023)
  4. FinGPT : Open-Source Financial Large Language Models (2023)
  5. Llama 2: Open foundation and fine-tuned chat models (2023) — link
  6. Med-halt: Medical domain hallucination test for large language models (2023) — link
  7. FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) — link
  8. Retrieval augmentation reduces hallucination in conversation (2021) — link
and 42 more
  1. Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection (2023) — link
  2. Language models (mostly) know what they know (2022) — link
  3. The internal state of an llm knows when its lying (2023) — link
  4. Check your facts and try again: Improving large language models with external knowledge and automated feedback (2023) — link
  5. Rarr: Researching and revising what language models say, using language models (2023)
  6. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023) — link
  7. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (2023) — link
  8. Can Chatgpt Improve Investment Decision? From a Portfolio Management Perspective (2023)
  9. Toolformer: Language models can teach themselves to use tools (2023) — link
  10. The curious case of neural text degeneration (2019) — link
  11. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models (2023) — link
  12. Instruct-FinGPT : Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models (2023)
  13. Chain-of-Verification Reduces Hallucination in Large Language Models (2023) — link
  14. Prompting gpt-3 to be reliable (2022) — link
  15. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation (2023) — link
  16. Billion-scale similarity search with gpus (2019)
  17. Retrieval-augmented generation for knowledge-intensive nlp tasks (2020)
  18. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023) — link
  19. Llama: Open and efficient foundation language models (2023) — link
  20. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023) — link
  21. PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
  22. Survey of hallucination in natural language generation (2023)
  23. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions (2023) — link
  24. Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2023) — link
  25. Contrastive decoding: Open-ended text generation as optimization (2022) — link
  26. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation (2023) — link
  27. LM vs LM: Detecting Factual Errors via Cross Examination (2023) — link
  28. Reducing conversational agents’ overconfidence through linguistic calibration (2022)
  29. Active retrieval augmented generation (2023) — link
  30. Do Language Models Know When They’re Hallucinating References? (2023) — link
  31. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (2023) — link
  32. React: Synergizing reasoning and acting in language models (2022) — link
  33. Gorilla: Large language model connected with massive apis (2023) — link
  34. Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering (2023) — link
  35. RARR : Researching and Revising What Language Models Say, Using Language Models (2023) — link
  36. Teaching language models to support answers with verified quotes (2022) — link
  37. Performance and exploration of ChatGPT in medical examination, records and education in Chinese: Pave the way for medical AI (2023)
  38. Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models (2023) — link
  39. Chatlaw: Open-source legal large language model with integrated external knowledge bases (2023) — link
  40. Language models are few-shot learners (2020)
  41. GPT-4 Technical Report (2023)
  42. Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027

1 of 50 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.