Paper: arXiv 2411.13599

Abstract

Large Language Models (LLMs) have achieved remarkable success recently, displaying exceptional capabilities in creating understandable and organized text. These LLMs have been utilized in diverse fields, such as clinical research, where domain-specific models like Med-Palm have achieved human-level performance. Recently, researchers have employed advanced prompt engineering to enhance the general reasoning ability of LLMs. Despite the remarkable success of zero-shot Chain-of-Thoughts (CoT) in solving general reasoning tasks, the potential of these methods still remains paid limited attention in the financial reasoning task.To address this issue, we explore multiple prompt strategies and incorporated semantic news information to improve LLMs’ performance on financial reasoning tasks.To the best of our knowledge, we are the first to explore this important issue by applying ChatGPT to the gold investment.In this work, our aim is to investigate the financial reasoning capabilities of LLMs and their capacity to generate logical and persuasive investment opinions. We will use ChatGPT, one of the most powerful LLMs recently, and prompt engineering to achieve this goal. Our research will focus on understanding the ability of LLMs in sophisticated analysis and reasoning within the context of investment decision-making. Our study finds that ChatGPT with CoT prompt can provide more explainable predictions and overcome behavioral biases, which is crucial in finance-related tasks and can achieve higher investment returns.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.5: transaction costs not modelled; no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper’s primary mathematics is basic (score normalization and correlation), but it demonstrates strong empirical rigor through a detailed back-testing framework with specific datasets, risk metrics (Sharpe ratio), and a clear trading strategy implementation.

Research Flowchart

  flowchart TD
  A["Research Goal: Can ChatGPT overcome behavioral biases in gold investment?"] --> B{"Methodology"}
  B --> C["Prompt Engineering"]
  B --> D["Zero-Shot Chain-of-Thought CoT"]
  B --> E["Multi-Step Reasoning"]
  F["Data: Gold Investment Scenarios & News"] --> B
  C & D & E --> G["Computational Process: Classify-and-Rethink"]
  G --> H["Outcome: Explainable Investment Decisions"]
  H --> I["Results: Reduced Biases & Higher Returns"]
  I --> J["Limitations: Model Ambiguity & Data Constraints"]

Cited References (81)

  1. The index of general nonlinear D A E S (1995)
  2. Clinical implications of dysregulated cytokine production (2000) — link
  3. Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations (1995)
  4. Algorithms for C omputer A lgebra (1992)
  5. Software engineering—from auxiliary to key technologies (1992)
  6. Conductive P olymers (1981)
  7. Neuromuscular blocking drugs in man (1976)
  8. Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae (1978)
and 73 more
  1. Global integrated drought monitoring and prediction system (GIDMaPS) data sets (2014)
  2. Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon (2002)
  3. Mixing induced CP asymmetries in inclusive B decays (1997) — link
  4. deep SIP : deep learning of S upernova I a P arameters (2020) — link
  5. Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing (2019) — link
  6. Beyond Classification: Financial Reasoning in State-of-the-Art Language Models (2023)
  7. The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges (2023)
  8. Chatdoctor: A medical chat model fine-tuned on llama model using medical domain knowledge (2023)
  9. Legal Prompt Engineering for Multilingual Legal Judgement Prediction (2022)
  10. Large Language Models Encode Clinical Knowledge (2022)
  11. Emergent abilities of large language models (2022)
  12. Progressive-hint prompting improves reasoning in large language models (2023)
  13. Llama: Open and efficient foundation language models (2023)
  14. GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022)
  15. Bloom: A 176b-parameter open-access multilingual language model (2022)
  16. WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain (2022)
  17. Finbert: Financial sentiment analysis with pre-trained language models (2019)
  18. Finetuned Language Models Are Zero-Shot Learners (2021)
  19. Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Finance (2023)
  20. Training language models to follow instructions with human feedback (2022)
  21. Deep reinforcement learning from human preferences (2017)
  22. BloombergGPT: A Large Language Model for Finance (2023)
  23. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023)
  24. Successive Prompting for Decomposing Complex Questions (2022)
  25. Automatic Chain of Thought Prompting in Large Language Models (2022)
  26. Large Language Models are Zero-Shot Reasoners (2022)
  27. LLM+P: Empowering Large Language Models with Optimal Planning Proficiency (2023)
  28. LaMDA: Language Models for Dialog Applications (2022)
  29. Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents (2023)
  30. Training Verifiers to Solve Math Word Problems (2021)
  31. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019)
  32. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies (2021)
  33. On the Advance of Making Language Models Better Reasoners (2022)
  34. Are NLP Models really able to Solve Simple Math Word Problems? (2021)
  35. Towards Reasoning in Large Language Models: A Survey (2022)
  36. Measuring Mathematical Problem Solving With the MATH Dataset (2021)
  37. Solving General Arithmetic Word Problems (2016)
  38. Large Language Models are reasoners with Self-Verification (2022)
  39. Measuring and Narrowing the Compositionality Gap in Language Models (2022)
  40. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (2022)
  41. Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022)
  42. Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2022)
  43. Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022)
  44. Parsing Algebraic Word Problems into Equations (2015)
  45. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks (2022)
  46. Towards a Unified View of Parameter-Efficient Transfer Learning (2021)
  47. PaLM: Scaling Language Modeling with Pathways (2022)
  48. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them (2022)
  49. Language Models are Few-Shot Learners (2020)
  50. Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (2022)
  51. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models (2022)
  52. Parameter-Efficient Transfer Learning for NLP (2019)
  53. Black-Box Tuning for Language-Model-as-a-Service (2022)
  54. Complexity-Based Prompting for Multi-Step Reasoning (2022)
  55. RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019)
  56. MAWPS: A Math Word Problem Repository (2016)
  57. Decomposed Prompting: A Modular Approach for Solving Complex Tasks (2022)
  58. On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2022)
  59. Learning to Solve Arithmetic Word Problems with Verb Categorization (2014)
  60. PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents (2023)
  61. Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems (2017)
  62. CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge (2019)
  63. Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning (2022)
  64. Tree of Thoughts: Deliberate Problem Solving with Large Language Models (2023)
  65. ReAct: Synergizing Reasoning and Acting in Language Models (2022)
  66. Structure and Interpretation of Computer Programs (1985)
  67. Visual Information Extraction with Lixto (2001)
  68. An overview of the KL-ONE knowledge representation system (1985)
  69. Complexity results for nonmonotonic logics (1992)
  70. Hypertree Decompositions and Tractable Queries (2002)
  71. Foundations of a functional approach to knowledge representation (1984)
  72. A logic of implicit and explicit belief (1984)
  73. On the compilability and expressive power of propositional planning formalisms (2000)

0 of 81 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.