Paper: arXiv 2609.35086

Authors: Kelvin J. L. Koa, Xinyang Li, Ke-Wei Huang

Abstract

In this work, we study portfolio optimization under the stochastic discount factor (SDF) framework by learning market state representations that capture the underlying risk structures of financial data. This is challenging due to several factors: financial markets exhibit non-stationary dynamics with shifting regimes, multimodal inputs such as price and news data often contain stochastic noise, and existing diffusion-based approaches, while effective for modeling stochastic dynamics, rely on assumptions such as isotropic Gaussian noise that fail to capture the state-dependent nature of financial uncertainty. To address these challenges, we introduce RADAR, a retrieval-augmented diffusion framework that learns market representations by conditioning on similar historical regimes. RADAR leverages retrieval to construct context-dependent noise distributions, applies conditional diffusion to denoise multimodal representations, and initializes the diffusion process using empirical statistics to reflect state-dependent uncertainty. Experiments show that RADAR achieves state-of-the-art performance on key risk-adjusted metrics while producing economically meaningful signals on asset returns and correlations.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a highly novel approach by integrating retrieval-augmented diffusion models for SDF-based portfolio optimization, addressing key challenges in financial modeling. It demonstrates strong empirical results with state-of-the-art performance and provides code for reproducibility, making it a significant contribution.

Research Flowchart

  flowchart TD
    A[Research Goal: Portfolio Optimization under SDF] --> B{Challenges: Non-stationary, Multimodal Noise, Isotropic Diffusion Limits};

    B --> C[Methodology: RADAR Framework];

    C --> D{Data/Inputs: Price Data, News Data};

    D --> E[Computational Processes: Retrieval-Augmented Diffusion, Conditional Diffusion, Empirical Statistics Init.];

    E --> F[Key Findings/Outcomes: SOTA Risk-Adjusted Metrics, Economically Meaningful Signals];

Cited References (59)

  1. The Utility of Wealth (1952) — link
  2. Portfolio Selection (1952) — link
  3. Large (and deep) factor models — → 2402.06635
  4. Artificial intelligence asset pricing models (2025)
  5. Deep attentive learning for stock movement prediction from social media text and company correlations (2020)
  6. Listening to chaotic whispers: A deep learning framework for news-oriented stock trend prediction (2018)
  7. Diffusion variational autoencoder for tackling stochasticity in multi-step regression stock price prediction — → 2309.00073
  8. A new approach to the economic analysis of nonstationary time series and the business cycle (1989)
and 51 more
  1. Giving content to investor sentiment: The role of media in the stock market (2007)
  2. Diffsformer: A diffusion transformer on stock factor augmentation — → 2402.06656
  3. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting (2021)
  4. Deep learning for event-driven stock prediction. (2015)
  5. A dual-stage attention-based recurrent neural network for time series prediction (2017) — link
  6. Efficient capital markets: A review of theory and empirical work (1970)
  7. Learning to generate explainable stock predictions using self-reflective large language models — → 2402.03659
  8. Stock movement prediction from tweets and historical prices (2018)
  9. A Random Walk Down Wall Street (1973)
  10. Enhancing stock movement prediction with adversarial training (2018) — link
  11. Building diversified portfolios that outperform out-of-sample (2016)
  12. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  13. Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks (2019)
  14. Learning embedded representation of the stock correlation matrix using graph machine learning (2024)
  15. Clustering and Similarity Learning in Financial Markets: A Tutorial for the Practitioners (2025)
  16. Asset embeddings (2025)
  17. The role of conditioning information in deducing testable restrictions implied by dynamic asset pricing models (1987)
  18. Honey, I shrunk the sample covariance matrix (2003)
  19. Finbert: Financial sentiment analysis with pre-trained language models (2019) — link
  20. NGAT: A Node-level Graph Attention Network for Long-term Stock Prediction (2025)
  21. itransformer: Inverted transformers are effective for time series forecasting (2023) — link
  22. Parametric portfolio policies: Exploiting characteristics in the cross-section of equity returns (2009)
  23. Apt or “aipt”? the surprising dominance of large factor models (2024)
  24. Digesting anomalies: An investment approach (2015)
  25. Mispricing factors (2017)
  26. Short-and long-horizon behavioral factors (2020)
  27. Fnspid: A comprehensive financial news dataset in time series (2024)
  28. Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? (2009)
  29. Fin-r1: A large language model for financial reasoning through reinforcement learning (2025) — link
  30. Time-mmd: Multi-domain multimodal dataset for time series analysis (2024)
  31. Prompt leakage effect and mitigation strategies for multi-turn LLM applications (2024)
  32. A unified approach to interpreting model predictions (2017)
  33. " Why should i trust you?" Explaining the predictions of any classifier (2016)
  34. A simple, positive semi-definite, heteroskedasticity and autocorrelationconsistent covariance matrix (1986)
  35. Comparing predictive accuracy (2002)
  36. Robust performance hypothesis testing with the Sharpe ratio (2008)
  37. The stationary bootstrap (1994)
  38. Statistical methods for research workers. (1934)
  39. Retrieval-augmented diffusion models for time series forecasting (2024)
  40. Pattern recognition and machine learning (2006)
  41. Machine learning: a probabilistic perspective (2012)
  42. Non-stationary transformers: Exploring the stationarity in time series forecasting (2022)
  43. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting (2021)
  44. Denoising diffusion probabilistic models (2020)
  45. Blurring Diffusion Models (2023)
  46. Diffusion models with learned adaptive noise (2024)
  47. Neural flow diffusion models: Learnable forward process for improved diffusion modelling (2024)
  48. Neural diffusion models (2023) — link
  49. Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting (2025)
  50. Non-stationary Diffusion For Probabilistic Time Series Forecasting (2025)
  51. Transformer-modulated diffusion models for probabilistic multivariate time series forecasting (2024)

4 of 59 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.