Paper: arXiv 2409.19854

Abstract

This paper proposes a novel method for constructing instruction-tuned large language models (LLMs) for finance without instruction data. Traditionally, developing such domain-specific LLMs has been resource-intensive, requiring a large dataset and significant computational power for continual pretraining and instruction tuning. Our study proposes a simpler approach that combines domain-specific continual pretraining with model merging. Given that general-purpose pretrained LLMs and their instruction-tuned LLMs are often publicly available, they can be leveraged to obtain the necessary instruction task vector. By merging this with a domain-specific pretrained vector, we can effectively create instruction-tuned LLMs for finance without additional instruction data. Our process involves two steps: first, we perform continual pretraining on financial data; second, we merge the instruction-tuned vector with the domain-specific pretrained vector. Our experiments demonstrate the successful construction of instruction-tuned LLMs for finance. One major advantage of our method is that the instruction-tuned and domain-specific pretrained vectors are nearly independent. This independence makes our approach highly effective. The Japanese financial instruction-tuned LLMs we developed in this study are available at https://huggingface.co/pfnet/nekomata-14b-pfn-qfin-inst-merge.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.5: no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The math complexity is low, as the paper focuses on a high-level methodology (model merging) without heavy mathematical derivations. The empirical rigor is high, evidenced by the construction of a specific Japanese financial corpus, execution of continual pretraining and model merging experiments, and the release of the resulting models on Hugging Face for reproducibility.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Instruction-tuned LLM for Finance<br>without Instruction Data"] --> B["Step 1: Continual Pretraining<br>Pretrain Base LLM on Financial Text"]
  B --> C["Step 2: Model Merging<br>Merge Domain LLM + Instruction Vector"]
  C --> D["Key Finding 1<br>Independence of Vectors<br>Effective without Joint Tuning"]
  C --> E["Key Finding 2<br>Successful Financial LLM<br>Released publicly on Hugging Face"]

Cited References (62)

  1. ChatGPT (2023)
  2. GPT-4 Technical Report (2023) — link
  3. A framework for few-shot language model evaluation (2021) — link
  4. Is it All Hype? ChatGPT’s Performance and Disruptive Potential in the Accounting and Auditing Industries (2023) — link
  5. Capabilities of GPT-4 on Medical Challenge Problems (2023)
  6. ChatGPT by OpenAI: The End of Litigation Lawyers? (2023) — link
  7. ChatGPT Goes to Law School (2023) — link
  8. Attention Is All You Need (2017)
and 54 more
  1. PaLM 2 Technical Report (2023)
  2. Llama 2: Open Foundation and Fine-Tuned Chat Models (2023)
  3. Improving Language Understanding by Generative Pre-Training (2018)
  4. Language Models are Unsupervised Multitask Learners (2019)
  5. Language Models are Few-Shot Learners (2020)
  6. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019) — link
  7. Bard (2023)
  8. Dolly (2023)
  9. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality (2023)
  10. LLaMA: Open and Efficient Foundation Language Models (2023)
  11. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022)
  12. PaLM: Scaling Language Modeling with Pathways (2022)
  13. BloombergGPT: A Large Language Model for Finance (2023)
  14. Fin-LLAMA: Efficient Finetuning of Quantized LLMs for Finance (2023)
  15. FinGPT: Open-Source Financial Large Language Models (2023)
  16. Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models (2023)
  17. From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models (2023)
  18. JMedLoRA: Medical Domain Adaptation on Japanese Large Language Models using Instruction-tuning (2023)
  19. llm-japanese-dataset v0: Construction of Japanese Chat Dataset for Large Language Models and its Methodology (2023) — link
  20. JP Language Model Evaluation Harness (2023)
  21. Gemini: a family of highly capable multimodal models (2023) — link
  22. Constructing and Analyzing Domain-Specific Language Model for Financial Text Mining (2023) — link
  23. llm-jp-eval (2024) — link
  24. Distributed Representations of Words and Phrases and their Compositionality (2013)
  25. Related Stocks Selection with Data Collaboration Using Text Mining (2019) — link
  26. Deep Contextualized Word Representations (2018) — link
  27. Bidirectional recurrent neural networks (1997) — link
  28. Universal Language Model Fine-tuning for Text Classification (2018) — link
  29. SciBERT: A Pretrained Language Model for Scientific Text (2019)
  30. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction (2021)
  31. Construction and Validation of a Pre-Training and Additional Pre-Training Financial Language Model [in Japanese] (2022) — link
  32. Don’t stop pretraining: Adapt language models to domains and tasks (2020) — link
  33. Lora: Low-rank adaptation of large language models (2021) — link
  34. Finetuned language models are zero-shot learners (2021) — link
  35. Deep reinforcement learning from human preferences (2017)
  36. Direct preference optimization: Your language model is secretly a reward model (2024)
  37. Lima: Less is more for alignment (2024)
  38. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters (2020)
  39. Accelerate: Training and inference at scale made simple, efficient and adaptable. (2022)
  40. Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling (2024) — link
  41. Can ChatGPT pass the JCPA exam?: Challenge for the short-answer method test on Auditing (2023)
  42. Deep model fusion: A survey (2023) — link
  43. Editing models with task arithmetic (2022) — link
  44. Evolutionary optimization of model merging recipes (2024) — link
  45. FinBERT: A large language model for extracting information from financial text (2023)
  46. Finbert: A pre-trained financial language representation model for financial text mining (2021)
  47. LLaMA-2-Econ: Enhancing Title Generation, Abstract Classification, and Academic Q&A in Economic Research (2024)
  48. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022)
  49. Ties-merging: Resolving interference when merging models (2024)
  50. Towards comprehensive and efficient post safety alignment of large language models via safety patching (2024) — link
  51. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet (2024)
  52. Construction of a Japanese Financial Benchmark for Large Language Models (2024) — link
  53. Judging llm-as-a-judge with mt-bench and chatbot arena (2024)
  54. pfmt-bench-fin-ja: Preferred Multi-turn Benchmark for Finance in Japanese (2024) — link

0 of 62 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.