Paper: arXiv 2409.05698

Abstract

It is widely acknowledged that extracting market sentiments from news data benefits market predictions. However, existing methods of using financial sentiments remain simplistic, relying on equal-weight and static aggregation to manage sentiments from multiple news items. This leads to a critical issue termed ``Aggregated Sentiment Homogenization’’, which has been explored through our analysis of a large financial news dataset from industry practice. This phenomenon occurs when aggregating numerous sentiments, causing representations to converge towards the mean values of sentiment distributions and thereby smoothing out unique and important information. Consequently, the aggregated sentiment representations lose much predictive value of news data. To address this problem, we introduce the Market Attention-weighted News Aggregation Network (MANA-Net), a novel method that leverages a dynamic market-news attention mechanism to aggregate news sentiments for market prediction. MANA-Net learns the relevance of news sentiments to price changes and assigns varying weights to individual news items. By integrating the news aggregation step into the networks for market prediction, MANA-Net allows for trainable sentiment representations that are optimized directly for prediction. We evaluate MANA-Net using the S&P 500 and NASDAQ 100 indices, along with financial news spanning from 2003 to 2018. Experimental results demonstrate that MANA-Net outperforms various recent market prediction methods, enhancing Profit & Loss by 1.1% and the daily Sharpe ratio by 0.252.

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced deep learning techniques like attention mechanisms and neural network architectures, indicating moderate-to-high math complexity, and it is empirically rigorous with a large 15-year dataset, backtested on S&P 500 and NASDAQ 100 indices, reporting specific PnL and Sharpe ratio improvements.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Enhance Market Prediction<br>using Financial News"] --> B["Identify Problem:<br>Aggregated Sentiment Homogenization"]
  B --> C["Data:<br>S&P 500/NASDAQ 100 & News 2003-2018"]
  C --> D["Method: MANA-Net<br>Dynamic Market-News Attention"]
  D --> E["Process: Learn Relevance &<br>Assign Varying Weights"]
  E --> F["Outcome: Trainable<br>Sentiment Representations"]
  F --> G["Key Findings:<br>+1.1% P&L & +0.252 Sharpe Ratio"]

Cited References (145)

  1. UK List of \TeX Frequently Asked Questions (2019)
  2. The \textsf amsart , \textsf amsproc , and \textsf amsbook document~classes (2004)
  3. a lgorithm2e.sty—package for algorithms (2015)
  4. The algorithms bundle (2009)
  5. The Listings Package (2015)
  6. Publication quality tables in \LaTeX (2005)
  7. ACM Visual Identity Standards (2007)
  8. The subcaption package (2013)
and 137 more
  1. A package to create a nomenclature (2005)
  2. User Manual for glossaries.sty v4.44 (2019)
  3. The \textsl textcase package (2004)
  4. Babel (2022)
  5. The Theory of Parsing, Translation and Compiling (1972)
  6. Publications Manual (1983)
  7. Alternation (1981) — link
  8. Scalable training of L1 -regularized log-linear models (2007)
  9. Algorithms on Strings, Trees and Sequences (1997)
  10. Yara Parser: A Fast and Accurate Dependency Parser (2015) — link
  11. Efficient capital markets a review of theory and empirical work (2021)
  12. Sentiment correlation in financial news networks and associated market movements (2021)
  13. When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks (2011)
  14. Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network (2013)
  15. Sentence-Level sentiment analysis of financial news using distributed text representations and multi-instance learning (2018) — link
  16. Big Data: Deep Learning for financial sentiment analysis (2018)
  17. Decision support from financial disclosures with deep neural networks and transfer learning (2017)
  18. Finbert: Financial sentiment analysis with pre-trained language models (2019) — link
  19. The role of text-extracted investor sentiment in Chinese stock price prediction with the enhancement of deep learning (2020)
  20. A hybrid model integrating deep learning with investor sentiment analysis for stock price prediction (2021)
  21. Roberta: A robustly optimized bert pretraining approach (2019) — link
  22. Contextual inter-modal attention for multi-modal sentiment analysis (2018)
  23. Attention-based multimodal fusion for video description (2017)
  24. Spanbert: Improving pre-training by representing and predicting spans (2020)
  25. Stock market prediction using LSTM recurrent neural network (2020)
  26. Predicting Stock Trends with CNN-BiLSTM Based Multi-Feature Integration Model (2021)
  27. Multi-modal attention network learning for semantic source code retrieval (2019) — link
  28. The efficient market hypothesis and its critics (2003)
  29. News impact on stock price return via sentiment analysis (2014)
  30. Textual analysis in accounting and finance: A survey (2016)
  31. Natural language based financial forecasting: a survey (2018)
  32. Financial sentiment analysis (fsa): A survey (2019)
  33. Network environment and financial risk using machine learning and sentiment analysis (2009)
  34. Crowds on wall street: Extracting value from social investing platforms (2014) — link
  35. Fine-grained sentiment analysis on financial microblogs and news headlines (2017)
  36. Financial sentiment lexicon analysis (2018)
  37. Twitter sentiment analysis with deep convolutional neural networks (2015)
  38. Deep learning for sentiment analysis: A survey (2018)
  39. Deep learning for stock market prediction from financial news articles (2017)
  40. Pretraining financial text encoder enhanced by lifelong learning (2020)
  41. Using text and data mining techniques to extract stock market sentiment from live news streams (2012)
  42. A Theoretic Framework Integrating Text Mining and Energy Demand Forecasting. (2007)
  43. Predicting stock price movements based on different categories of news articles (2015)
  44. Directional prediction of stock prices using breaking news on twitter (2015)
  45. Does summarization help stock prediction? A news impact analysis (2015)
  46. Stock market prediction using financial news articles on ho chi minh stock exchange (2016)
  47. Textual analysis of stock market prediction using breaking financial news: The AZFin text system (2009)
  48. R by example: Mining Twitter for consumer attitudes towards airlines (2011)
  49. Internet investment advice: Investing with a rock of salt (2003)
  50. Realization of a news dissemination agent based on weighted association rules and text mining techniques (2010)
  51. The information content of forward-looking statements in corporate filings—A na "\i ve Bayesian machine learning approach (2010)
  52. Combining the wisdom of crowds and technical analysis for financial market prediction using deep random subspace ensembles (2018)
  53. Stock price prediction using news sentiment analysis (2019)
  54. Forex-foreteller: Currency trend modeling using news articles (2013)
  55. Neural machine translation by jointly learning to align and translate (2014) — link
  56. Attention is all you need (2017)
  57. Deformable detr: Deformable transformers for end-to-end object detection (2020) — link
  58. The exchange rate effects of macro news after the global financial crisis (2019)
  59. All that glitters: The effect of attention and news on the buying behavior of individual and institutional investors (2008)
  60. Media sentiment and trading strategies of different types of traders (2017)
  61. Investor sentiment and analysts’ earnings forecast errors (2012)
  62. Relative importance of scheduled macroeconomic news for stock market investors (2003)
  63. Popular news are relevant news! How investor attention affects algorithmic decision-making and decision support in financial markets (2021)
  64. FAST: Financial news and tweet based time aware network for stock trading (2021)
  65. Applied attention-based LSTM neural networks in stock prediction (2018)
  66. Deep learning with long short-term memory networks for financial market predictions (2018)
  67. Deep learning stock volatility with google domestic trends (2015) — link
  68. Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500 (2017)
  69. Applications of news analytics in finance: A review (2011)
  70. Deep learning for financial sentiment analysis on finance news providers (2016)
  71. Applications of deep learning in stock market prediction: recent progress (2021)
  72. The right to tell: the role of mass media in economic development (2002)
  73. Giving content to investor sentiment: The role of media in the stock market (2007)
  74. How news and its context drive risk and returns around the world (2019)
  75. News dissemination and investor attention (2017)
  76. Macro news and micro news: complements or substitutes? (2022)
  77. Learning representations by back-propagating errors (1986)
  78. Approximation by superpositions of a sigmoidal function (1989)
  79. Deep learning using rectified linear units (relu) (2018) — link
  80. Equity2vec: End-to-end deep learning framework for cross-sectional asset pricing (2021)
  81. A deep learning framework for pricing financial instruments (2019)
  82. Backpropagation applied to handwritten zip code recognition (1989)
  83. Learning internal representations by error propagation (1985)
  84. Leveraging financial news for stock trend prediction with attention-based recurrent neural network (2018) — link
  85. Macroeconomics news and the stock market: Evidence from Europe (1989)
  86. Stock market overreactions to bad news in good times: a rational expectations equilibrium model (1999)
  87. Harvesting social media sentiment analysis to enhance stock market prediction using deep learning (2021)
  88. Stock price reaction to news: The joint effect of tone and attention on momentum (2017)
  89. Simplified gating in long short-term memory (lstm) recurrent neural networks (2017)
  90. Empirical evaluation of gated recurrent neural networks on sequence modeling (2014) — link
  91. Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014) — link
  92. How markets process information: News releases and volatility (1993)
  93. An empirical exploration of recurrent network architectures (2015)
  94. Holt’s exponential smoothing and neural network models for forecasting interval-valued time series (2011)
  95. Mixture of MLP-experts for trend forecasting of time series: A case study of the Tehran stock exchange (2011)
  96. Comparing the effectiveness of deep feedforward neural networks and shallow architectures for predicting stock price indices (2020)
  97. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  98. Bert: Pre-training of deep bidirectional transformers for language understanding (2019)
  99. Modeling the momentum spillover effect for stock prediction via attribute-driven graph attention networks (2021)
  100. Incorporating expert-based investment opinion signals in stock prediction: A deep learning framework (2020)
  101. Evaluation of sentiment analysis in finance: from lexicons to transformers (2020)
  102. The volatility of the stock market and news (2007)
  103. Dynamic auto-encoders for semantic indexing (2010)
  104. Deep learning for event-driven stock prediction (2015)
  105. Stock selection via spatiotemporal hypergraph attention network: A learning to rank approach (2021)
  106. Deep switching auto-regressive factorization: Application to time series forecasting (2021)
  107. The stock sonar—sentiment analysis of stocks based on a hybrid approach (2011)
  108. The role of machine learning in business optimization (2010)
  109. Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
  110. Automated news reading: Stock price prediction based on financial news using context-capturing features (2013)
  111. Weblog analysis for predicting correlations in stock price evolutions (2012)
  112. Factors affecting text mining based stock prediction: Text feature representations, machine learning models, and news platforms (2022)
  113. Transformer-based attention network for stock movement prediction (2022)
  114. Using financial news sentiment for stock price direction prediction (2022)
  115. A novel ensemble deep learning model for stock prediction based on stock prices and news (2022)
  116. LSTM based stock prediction using weighted and categorized financial news (2023)
  117. Multi-source aggregated classification for stock price movement prediction (2023)
  118. Temporal relational ranking for stock prediction (2019)
  119. Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models (2023)
  120. A sentiment analysis approach to the prediction of market volatility (2022)
  121. Knowledge-driven stock trend prediction and explanation via temporal convolutional network (2019)
  122. Online knowledge distillation with diverse peers (2020)
  123. Reducing Cross-Topic Political Homogenization in Content-Based News Recommendation (2022)
  124. One fits all: Power general time series analysis by pretrained lm (2023)
  125. Transformer-based capsule network for stock movement prediction (2019)
  126. Stock market index prediction using deep Transformer model (2022)
  127. Cost of capital (2008)
  128. A multimodal event-driven LSTM model for stock prediction using online news (2020)
  129. Transformation of dense and sparse text representations (2019) — link
  130. Hrn: A holistic approach to one class learning (2020)
  131. Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction — → 2309.00073
  132. Stock trend prediction with multi-granularity data: A contrastive learning approach with adaptive fusion (2021)
  133. Temporal and heterogeneous graph neural network for financial time series prediction — → 2305.08740
  134. Price forecast with high-frequency finance data: An autoregressive recurrent neural network model with technical indicators (2020)
  135. Hierarchical Multi-Scale Gaussian Transformer for Stock Movement Prediction. (2020)
  136. Efficient capital markets: A review of theory and empirical work (1970)
  137. Market reaction to internet news: Information diffusion and price pressure (2016)

2 of 145 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.