Paper: arXiv 2412.07587

Abstract

This article introduces a Hype-Adjusted Probability Measure in the context of a new Natural Language Processing (NLP) approach for stock return and volatility forecasting. A novel sentiment score equation is proposed to represent the impact of intraday news on forecasting next-period stock return and volatility for selected U.S. semiconductor tickers, a very vibrant industry sector. This work improves the forecast accuracy by addressing news bias, memory, and weight, and incorporating shifts in sentiment direction. More importantly, it extends the use of the remarkable tool of change of Probability Measure developed in the finance of Asset Pricing to NLP forecasting by constructing a Hype-Adjusted Probability Measure, obtained from a redistribution of the weights in the probability space, meant to correct for excessive or insufficient news.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel Hype-Adjusted Probability Measure using Radon-Nikodym derivatives and conditional expectations (high math) and presents a complete backtest with linear discriminant analysis, accuracy metrics, and data sources like LSEG/Eikon (high empirical rigor).

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Improve NLP Stock<br>Return & Volatility<br>Forecasting"] --> B["Key Methodology:<br>1. Novel Sentiment Score<br>2. Hype-Adjusted Probability<br>Measure (HAPM)"]
  B --> C["Data Inputs:<br>Selected U.S.<br>Semiconductor Tickers<br>Time-Series Data"]
  C --> D["Computational Processes:<br>Process News (Bias/Memory/Weight)<br>Calculate Sentiment Shifts<br>Redistribute Probability Weights<br>via Change of Measure"]
  D --> E["Key Outcomes:<br>1. Improved Forecast Accuracy<br>2. Corrects News Bias/Excess<br>3. Extends Finance Tools<br>to NLP"]

Cited References (32)

  1. A sentiment analysis approach to the prediction of market volatility (2022) — link
  2. The Pricing of Options and Corporate Liabilities (1973) — link
  3. Master’s Thesis: The Hyped-Adjusted Probability Measure (2025)
  4. LSEG Data and Analytics (2024) — link
  5. Introduction to News Sentiment Analysis with Eikon Data APIs - a Python example (2024) — link
  6. Linear Discriminant Analysis: A Brief Tutorial (1998) — link
  7. Financial News Data (2022) — link
  8. Financial Reports SEC (2022) — link
and 24 more
  1. EDGAR Crawler (2022) — link
  2. Financial News Headlines (2022) — link
  3. Monte Carlo Methods in Financial Engineering (2004)
  4. A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (1993)
  5. Arbitrage-Free SVI Volatility Surfaces (2014)
  6. Honors Senior Thesis: Stochastic Calculus for Option Pricing with Convex Duality, Logistic Model, & Numerical Examination (2023) — link
  7. Theoretical and Empirical Validation of Heston Model (2024) — link
  8. Convex Duality in Continuous Option Pricing Models (2024) — link
  9. Martingales and arbitrage in multiperiod securities markets (1979) — link
  10. The Theory of Speculation (1900) — link
  11. Changes of Num' e raire, Changes of Probability Measure and Option Pricing (1995) — link
  12. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (2000)
  13. Sentiment Analysis and Opinion Mining (2012) — link
  14. VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text (2014) — link
  15. Driven to Distraction: Extraneous Events and Underreaction to Earnings News (2009) — link
  16. Investor Sentiment and the Cross-Section of Stock Returns (2006) — link
  17. Sentiment-Guided Adversarial Learning for Stock Price Prediction (2021)
  18. New News is Bad News — → 2309.05560
  19. A Neural Probabilistic Language Model (2003)
  20. Twitter sentiment around the earnings announcement events (2016) — link
  21. Does Unusual News Forecast Market Stress? (2019)
  22. Measuring News Sentiment (2020) — link
  23. The Importance of the Forward Measure for the Pricing of Interest Rate Derivatives (1989)
  24. Proof That Properly Anticipated Prices Fluctuate Randomly (1965)

1 of 32 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.