Paper: arXiv 2312.16637

Abstract

We use the statistical properties of Shannon entropy estimator and Kullback-Leibler divergence to study the predictability of ultra-high frequency financial data. We develop a statistical test for the predictability of a sequence based on empirical frequencies. We show that the degree of randomness grows with the increase of aggregation level in transaction time. We also find that predictable days are usually characterized by high trading activity, i.e., days with unusually high trading volumes and the number of price changes. We find a group of stocks for which predictability is caused by a frequent change of price direction. We study stylized facts that cause price predictability such as persistence of order signs, autocorrelation of returns, and volatility clustering. We perform multiple testing for sub-intervals of days to identify whether there is predictability at a specific time period during the day.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 4.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper is highly mathematical, featuring rigorous statistical derivations of entropy estimators, Neyman-Pearson tests, and asymptotic distributions (e.g., χ²), but it lacks implementation details, backtest results, or transaction cost analysis, making it more theoretical than backtest-ready.

Research Flowchart

  flowchart TD
  A["Research Goal: Test for price predictability<br>in ultra-high frequency data"] --> B["Methodology: Shannon Entropy &<br>Kullback-Leibler Divergence"]
  B --> C["Data Input: Ultra-High Frequency<br>Equity Transaction Data"]
  C --> D["Computational Process:<br>1. Calculate Entropy at various<br>aggregation levels<br>2. Perform statistical tests<br>3. Analyze stylized facts<br>(order signs, autocorrelation, volatility)"]
  D --> E["Key Findings:<br>- Randomness increases with aggregation<br>- Predictable days = high trading activity<br>- Persistence in order flow causes predictability<br>- Predictability varies by time of day"]

Cited References (65)

  1. The subtle nature of financial random walks (2005) — link
  2. CHAPTER 2 - How Markets Slowly Digest Changes in Supply and Demand (2009) — link
  3. The Econometrics of Ultra-High-Frequency Data (2000) — link
  4. The Long Memory of the Efficient Market (2004) — link
  5. Statistical properties of stock order books: empirical results and models (2002) — link
  6. Autoregressive Conditional Duration: A New Model for Irregularly Spaced Transaction Data (1998) — link
  7. The Stochastic Dependence of Security Price Changes and Transaction Volumes: Implications for the Mixture-of-Distributions Hypothesis (1976) — link
  8. Fluctuations and response in financial markets: the subtle nature of `random’ price changes (2003) — link
and 57 more
  1. Leverage Effect in Financial Markets: The Retarded Volatility Model (2001) — link
  2. An Empirical Analysis of the Limit Order Book and the Order Flow in the Paris Bourse (1995) — link
  3. A New Approach for the Dynamics of Ultra-High-Frequency Data: The Model with Uncertainty Zones (2010) — link
  4. Limit Order Book Markets (2013) — link
  5. Limit order books (2013) — link
  6. Limit Distributions for a Statistical Estimate of the Entropy (1974) — link
  7. A statistical test of market efficiency based on information theory (2023) — link
  8. Rectangular Confidence Regions for the Means of Multivariate Normal Distributions (1967) — link
  9. Sampling returns for realized variance calculations: tick time or transaction time? (2008) — link
  10. Properties of Realized Variance Under Alternative Sampling Schemes (2006) — link
  11. Entropy and the Consistent Estimation of Joint Distributions (1994) — link
  12. A mathematical theory of communication (1948) — link
  13. Statistical Methods in Markov Chains (1961)
  14. The frequency goodness of fit test for probability chains (1951)
  15. The Likelihood Test of Independence in Contingency Tables (1935)
  16. The Ergodic Theory of Discrete Sample Paths (1996)
  17. On Information and Sufficiency (1951) — link
  18. Entropy estimation of symbol sequences (1996) — link
  19. On a Statistical Estimate for the Entropy of a Sequence of Independent Random Variables (1959) — link
  20. Jumps in Financial Markets: A New Nonparametric Test and Jump Dynamics (2007) — link
  21. Fact or friction: Jumps at ultra high frequency (2014) — link
  22. Event studies: A methodology review (2011) — link
  23. Local order, entropy and predictability of financial time series (2000) — link
  24. Stock market uncertainty and economic fundamentals: an entropy-based approach (2019) — link
  25. Entropy and Efficiency of the ETF Market (2020) — link
  26. The Dynamics of Market Efficiency (2016) — link
  27. The adaptive market hypothesis in the high frequency cryptocurrency market (2019) — link
  28. High frequency trading, price discovery and market efficiency in the FTSE100 (2019) — link
  29. Technical trading and cryptocurrencies (2021) — link
  30. Technical trading: Is it still beating the foreign exchange market? (2016) — link
  31. Forecasting high-frequency stock returns: A comparison of alternative methods (2022) — link
  32. Automatic trading method based on piecewise aggregate approximation and multi-swarm of improved self-adaptive particle swarm optimization with validation (2017) — link
  33. Does Academic Research Destroy Stock Return Predictability? (2016) — link
  34. Efficient Capital Markets: A Review of Theory and Empirical Work (1970) — link
  35. A non-parametric test for independence based on symbolic dynamics (2007) — link
  36. Irregularity, volatility, risk, and financial market time series (2004) — link
  37. Market efficiency in foreign exchange markets (2007) — link
  38. A singular value decomposition entropy approach for testing stock market efficiency (2021) — link
  39. On the use and interpretation of certain test criteria for purposes of statistical inference (1928) — link
  40. The valuation effects of stock splits and stock dividends (1984) — link
  41. Stock price reaction to news and no-news: drift and reversal after headlines (2003) — link
  42. Stock Price Reaction to News: The Joint Effect of Tone and Attention on Momentum (2017) — link
  43. Market efficiency in real time (2002) — link
  44. The Behavior of Stock Prices Around Institutional Trades (1995) — link
  45. Long memory and data frequency in financial markets (2019) — link
  46. Theory for long memory in supply and demand (2005)
  47. Estimating a Structural Model of Herd Behavior in Financial Markets (2014)
  48. The impact of institutional trading on stock prices (1992)
  49. A simulation analysis of the microstructure of double auction markets (2002)
  50. Long-memory in an order-driven market (2007)
  51. Foundations of technical analysis: Computational algorithms, statistical inference, and empirical implementation (2000)
  52. What do we know about the profitability of technical analysis? (2007)
  53. The adaptive markets hypothesis: Market efficiency from an evolutionary perspective (2004)
  54. Quantitative trading: algorithms, analytics, data, models, optimization (2017)
  55. Variance of entropy for testing time-varying regimes with an application to meme stocks (2024)
  56. Continuous auctions and insider trading (1985)
  57. Causes of fragile stock market stability (2022) — link

0 of 65 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.