Paper: arXiv 2403.00772

Abstract

Sentiment analysis can be used for stock market prediction. However, existing research has not studied the impact of a user’s financial background on sentiment-based forecasting of the stock market using artificial neural networks. In this work, a novel combination of neural networks is used for the assessment of sentiment-based stock market prediction, based on the financial background of the population that generated the sentiment. The state-of-the-art language processing model Bidirectional Encoder Representations from Transformers (BERT) is used to classify the sentiment and a Long-Short Term Memory (LSTM) model is used for time-series based stock market prediction. For evaluation, the Weibo social networking platform is used as a sentiment data collection source. Weibo users (and their comments respectively) are divided into Authorized Financial Advisor (AFA) and Unauthorized Financial Advisor (UFA) groups according to their background information, as collected by Weibo. The Hong Kong Hang Seng index is used to extract historical stock market change data. The results indicate that stock market prediction learned from the AFA group users is 39.67% more precise than that learned from the UFA group users and shows the highest accuracy (87%) when compared to existing approaches.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper relies primarily on established deep learning architectures (BERT, LSTM) with minimal novel mathematical formulation, but demonstrates strong empirical rigor through a large-scale, real-world dataset from Weibo, detailed data processing pipelines, public code availability, and clear comparative results against baseline methods.

Research Flowchart

  flowchart TD
  Start["Research Goal: Do financial experts perform better at stock market prediction?"] --> Data["Data: Weibo Comments &<br>Hong Kong Hang Seng Index"]
  Data --> BERT["Process: BERT Model<br>Classify Sentiment"]
  BERT --> Groups["Segment Users:<br>Authorized Financial Advisors (AFA)<br>Unauthorized Financial Advisors (UFA)"]
  Groups --> LSTM["Process: LSTM Model<br>Time-series Prediction"]
  LSTM --> Compare["Comparison & Evaluation"]
  Compare --> Outcomes["Outcomes: <br>AFA Predictions 39.67% more precise than UFA<br>Peak Accuracy: 87%"]

Cited References (57)

  1. Predicting the Future with Social Media (2010)
  2. Stock prediction using twitter sentiment analysis (2012)
  3. Sentiment analysis on social media for stock movement prediction (2015)
  4. Exploiting topic based twitter sentiment for stock prediction (2013)
  5. Stock Market Trend Prediction with Sentiment Analysis based on LSTM Neural Network (2019)
  6. Twitter mood predicts the stock market (2011)
  7. Stock market forecasting with financial micro-blog based on sentiment and time series analysis (2017)
  8. A nonlinear Granger causality test between stock returns and investor sentiment for Chinese stock market: a wavelet-based approach (2016)
and 49 more
  1. Can investor sentiment be used to predict the stock price? Dynamic analysis based on China stock market (2017)
  2. News impact on stock price return via sentiment analysis (2014)
  3. Has microblogging changed stock market behavior? Evidence from China (2016)
  4. Predicting Stock Price Returns Using Microblog Sentiment for Chinese Stock Market (2017)
  5. A LSTM-based method for stock returns prediction: A case study of China stock market (2015)
  6. LSTM Neural Network with Emotional Analysis for prediction of stock price. (2017)
  7. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  8. BERT-based Financial Sentiment Index and LSTM-based Stock Return Predictability (2019) — link
  9. Long short-term memory (1997)
  10. Modeling public mood and emotion: Stock market trend prediction with anticipatory computing approach (2019)
  11. Investor sentiment and the near-term stock market (2004)
  12. Pre-trained Models for Natural Language Processing: A Survey (20200318)
  13. A Review of Sentiment Analysis Research in Chinese Language (2017)
  14. Lexicon-based methods for sentiment analysis (2011)
  15. Building a Concept-Level Sentiment Dictionary Based on Commonsense Knowledge (2013)
  16. Efficient Estimation of Word Representations in Vector Space (2013)
  17. Review of Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition by Daniel Jurafsky and James H. Martin. Prentice Hall 2000. (2000)
  18. More Than Words: Quantifying Language to Measure Firms’ Fundamentals (2008) — link
  19. NewsCATS: A News Categorization and Trading System (2006)
  20. Daily stock market forecast from textual web data (1998) — link
  21. dataabc/weibo-search — link
  22. tracy-talent/curriculum — [link](https://github.com/tracy-talent/curriculum/tree/master/Data Mining/sentiment)
  23. google-research/bert — link
  24. SophonPlus/ChineseNlpCorpus — link
  25. rguthrie3/DeepLearningForNLPInPytorch — link
  26. Investing.com_Global financial market information expert: foreign exchange, stocks, futures, bonds, digital currency market and financial news — link
  27. Stock trend forecasting method based on sentiment analysis and system similarity model (2011)
  28. Attention-based LSTM for Aspect-level Sentiment Classification (2016)
  29. Influence of Investor Sentiment on Stock Returns- Analysis of A- shares in China (2018) — link
  30. Expected stock returns and volatility (1987)
  31. Stock price synchronicity and stock price crash risk: Based on the mediating effect of herding behavior of QFII (2016)
  32. An Empirical Study on the Herding Effect in China’s Stock Market (2004)
  33. Weibo customer service — link
  34. Building emotional dictionary for sentiment analysis of online news (2014)
  35. Designing a neural network for forecasting financial and economic time series (1996) — link
  36. pytorch — link
  37. Tweets and trades: The information content of stock microblogs (2014)
  38. How to fine-tune bert for text classification? (2019)
  39. LSTM can solve hard long time lag problems (1997)
  40. Financial time series forecasting model based on CEEMDAN and LSTM (2019)
  41. Impact of Training Dataset Size on Neural Answer Selection Models (2019)
  42. Sentiment Analysis on Weibo Platform for Stock Prediction (2020)
  43. Weibo sentiments and stock return: A time-frequency view (2017) — link
  44. Event-based user classification in weibo media (2014)
  45. Encyclopedia of machine learning and data mining (2017)
  46. Adversarial training for aspect-based sentiment analysis with BERT (2020) — link
  47. Audio albert: A lite bert for self-supervised learning of audio representation (2021)
  48. Learning to forget: Continual prediction with LSTM (1999)
  49. Can online emotions predict the stock market in China? (2016)

0 of 57 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.