Paper: arXiv 2405.20094

Abstract

Predicting the conditional evolution of Volterra processes with stochastic volatility is a crucial challenge in mathematical finance. While deep neural network models offer promise in approximating the conditional law of such processes, their effectiveness is hindered by the curse of dimensionality caused by the infinite dimensionality and non-smooth nature of these problems. To address this, we propose a two-step solution. Firstly, we develop a stable dimension reduction technique, projecting the law of a reasonably broad class of Volterra process onto a low-dimensional statistical manifold of non-positive sectional curvature. Next, we introduce a sequentially deep learning model tailored to the manifold’s geometry, which we show can approximate the projected conditional law of the Volterra process. Our model leverages an auxiliary hypernetwork to dynamically update its internal parameters, allowing it to encode non-stationary dynamics of the Volterra process, and it can be interpreted as a gating mechanism in a mixture of expert models where each expert is specialized at a specific point in time. Our hypernetwork further allows us to achieve approximation rates that would seemingly only be possible with very large networks.

Complexity vs Empirical Score

  • Math Complexity: 9.5/10
  • Empirical Rigor: 4.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper presents dense, advanced mathematics including Riemannian geometry, non-positive curvature, and universal approximation theorems for Volterra processes, with minimal empirical validation limited to an ablation study without backtests or real financial data.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Predict conditional evolution<br>of Volterra processes"] --> B{"Key Challenge<br>Curse of dimensionality<br>from infinite, non-smooth paths"}
  B --> C["Step 1: Dimensionality Reduction<br>Project law to low-dim manifold<br>of non-positive curvature"]
  C --> D["Step 2: Sequential DL Model<br>Geometry-aware architecture"]
  D --> E["Hypernetwork & Gating<br>Dynamically updates parameters<br>Mixture of experts for time"]
  E --> F["Computational Process<br>Train model on manifold<br>to approximate conditional law"]
  F --> G["Key Findings/Outcomes<br>Stable reduction<br>High approximation rates<br>without large networks"]

Cited References (149)

  1. Global universal approximation of functional input maps on weighted spaces — → 2306.03303
  2. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems (1995)
  3. Two-layer neural networks with values in a Banach space (2022)
  4. Neural networks in Fr 'e chet spaces (2023)
  5. Patch-level routing in mixture-of-experts is provably sample-efficient for convolutional neural networks (2023)
  6. Mixture of Experts Soften the Curse of Dimensionality in Operator Learning (2024) — link
  7. Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts (2024) — link
  8. Self-Instruct: Aligning Language Models with Self-Generated Instructions (2023) — link
and 141 more
  1. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019) — link
  2. Introducing DBRX: A New State-of-the-Art Open LLM (2024)
  3. Active ranking of experts based on their performances in many tasks (2023)
  4. Merge, Then Compress: Demystify Efficient SM oE with Hints from Its Routing Policy (2024) — link
  5. From Sparse to Soft Mixtures of Experts (2024) — link
  6. Adaptive mixtures of local experts (1991)
  7. Hierarchical mixtures of experts and the EM algorithm (1994)
  8. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer (2017) — link
  9. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity (2022)
  10. Mixture-of-experts meets instruction tuning: A winning combination for large language models (2023) — link
  11. Mixture-of-experts with expert choice routing (2022)
  12. Megablocks: Efficient sparse training with mixture-of-experts (2023)
  13. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models (2024) — link
  14. Mixtral of experts (2024) — link
  15. Mistral 7B (2023) — link
  16. Barycenters in Alexandrov spaces of curvature bounded below (2012)
  17. Universal Approximation Under Constraints is Possible with Transformers (2022) — link
  18. Simultaneous linear connectivity of neural networks modulo permutation (2024) — link
  19. Git Re-Basin: Merging Models modulo Permutation Symmetries (2023) — link
  20. An exact mapping from ReLU networks to spiking neural networks (2023)
  21. Expressivity of Spiking Neural Networks (2023) — link
  22. Approximation spaces of deep neural networks (2022)
  23. Algebraic topology (1989)
  24. Wasserstein Riemannian geometry of Gaussian densities (2018)
  25. Riemannian Lp center of mass: existence, uniqueness, and convexity (2011)
  26. From trees to continuous embeddings and back: Hyperbolic hierarchical clustering (2020)
  27. Latent graph inference using product manifolds (2022)
  28. On the computation of Wasserstein barycenters (2020)
  29. Fast computation of Wasserstein barycenters (2014)
  30. Estimating Barycenters of Distributions with Neural Optimal Transport (2024) — link
  31. Energy-Guided Continuous Entropic Barycenter Estimation for General Costs (2024) — link
  32. Geodesic convolutional neural networks on riemannian manifolds (2015)
  33. Intrinsic shape context descriptors for deformable shapes (2012)
  34. Semi-Supervised Classification with Graph Convolutional Networks (2017) — link
  35. A general version of the fundamental theorem of asset pricing (1994)
  36. Wasserstein geometry of Gaussian measures (2011)
  37. A geometric study of Wasserstein spaces: Euclidean spaces (2010)
  38. Exponential relu neural network approximation rates for point and edge singularities (2022)
  39. Approximation rates for neural networks with encodable weights in smoothness spaces (2021)
  40. Reservoir computing approaches to recurrent neural network training (2009)
  41. Fading memory echo state networks are universal (2021)
  42. Approximation and Optimization Theory for Linear Continuous-Time Recurrent Neural Networks. (2022)
  43. Wasserstein generative adversarial networks (2017)
  44. HyperNetworks (2017) — link
  45. Deep unsupervised learning using nonequilibrium thermodynamics (2015)
  46. The R iemannian geometry of the space of positive-definite matrices and its application to the regularization of positive-definite matrix-valued data (2011)
  47. General Signature Kernels (2021) — link
  48. Adapted topologies and higher rank signatures (2020) — link
  49. Riemannian batch normalization for SPD neural networks (2019)
  50. An Accelerated First-Order Method for Non-convex Optimization on Manifolds (2022) — link
  51. Fisher-Rao geometry of Dirichlet distributions (2021)
  52. Riemannian geometry (1990)
  53. On a formula for the L2 Wasserstein metric between measures on Euclidean and Hilbert spaces (1990)
  54. Convex Analysis and Monotone Operator Theory in Hilbert Spaces (2017)
  55. Cot-gan: Generating sequential data via causal optimal transport (2020)
  56. Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer (2023)
  57. Geodesic regression and the theory of least squares on Riemannian manifolds (2013)
  58. Deep arbitrage-free learning in a generalized HJM framework via arbitrage-regularization (2020)
  59. A Transfer Principle: Universal Approximators Between Metric Spaces From Euclidean Universal Approximators (2023) — link
  60. Small Transformers Compute Universal Metric Embeddings (2023) — link
  61. The matrix cookbook (2008)
  62. Conditional time series forecasting with convolutional neural networks (2017) — link
  63. Principal geodesic analysis for the study of nonlinear statistics of shape (2004)
  64. Locating Fr 'e chet means with application to shape spaces (2001)
  65. Theory of rational option pricing (1973)
  66. Polynomial Volterra processes (2024) — link
  67. From Bachelier to Dupire via optimal transport (2022)
  68. The pricing of options and corporate liabilities (1973)
  69. Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality (2023)
  70. Nonpositive curvature, the variance functional, and the Wasserstein barycenter (2020)
  71. Differentially Private Fr'echet Mean on the Manifold of Symmetric Positive Definite ( SPD ) Matrices with log-Euclidean Metric (2023) — link
  72. Scalable Bayes via barycenter in Wasserstein space (2018)
  73. Dilated convolutional neural networks for time series forecasting (2018)
  74. Riemannian center of mass and mollifier smoothing (1977)
  75. How to conjugate C 1-close group actions (1973)
  76. Are Transformers universal approximators of sequence-to-sequence functions? (2020) — link
  77. Reservoir computing universality with stochastic inputs (2019)
  78. Modern portfolio theory, 1950 to date (1997)
  79. Universal approximation theorems for differentiable geometric deep learning (2022)
  80. Non-euclidean universal approximation (2020)
  81. A Riemannian framework for tensor computing (2006)
  82. Turing Completeness of Bounded-Precision Recurrent Neural Networks (2021) — link
  83. On the Turing Completeness of Modern Neural Network Architectures (2019) — link
  84. Approximation and Optimization Theory for Linear Continuous-Time Recurrent Neural Networks. (2022)
  85. Infinite-dimensional reservoir computing (2023) — link
  86. On the difficulty of training recurrent neural networks (2013)
  87. Portfolio selection: efficient diversification of investments (1968)
  88. Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis (2022) — link
  89. Efficient approximation of high-dimensional functions with neural networks (2021)
  90. Multi-Scale Message Passing Neural PDE Solvers (2023) — link
  91. Universal Regular Conditional Distributions (2023)
  92. Error bounds for approximations with deep ReLU networks (2017)
  93. Optimal approximation of continuous functions by very deep ReLU networks (2018) — link
  94. The phase diagram of approximation rates for deep neural networks (2020)
  95. Discrete-time signal processing. Vol. 2 (2001)
  96. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition (1982)
  97. Theory of deep convolutional neural networks: Downsampling (2020) — link
  98. Backpropagation applied to handwritten zip code recognition (1989)
  99. Ueber die kleinste Kugel, die eine räumliche Figur einschliesst. (1901) — link
  100. The Parametric Complexity of Operator Learning (2023) — link
  101. Exponential ReLU DNN expression of holomorphic maps in high dimension (2022)
  102. Nonlinear filtering in riemannian manifolds (1984)
  103. Feedback particle filter on riemannian manifolds and matrix lie groups (2017)
  104. Particle filtering on Riemannian manifolds (2006)
  105. Random Toeplitz matrices: The condition number under high stochastic dependence (2021)
  106. Exponential-Wrapped Distributions on Symmetric Spaces (2022) — link
  107. Instance-Dependent Generalization Bounds via Optimal Transport (2022) — link
  108. Locally flat imbeddings of topological manifolds (1962)
  109. Bregman-Wasserstein divergence: geometry and applications (2023) — link
  110. Ueber den Begriff der vollst "a ndigen differentialgeometrischen Fl "a che (1931)
  111. Geodesic convolutional neural networks on riemannian manifolds (2015)
  112. Geometrical foundations of asymptotic inference (2011)
  113. Deep learning (2016)
  114. A fast learning algorithm for deep belief nets (2006)
  115. Conditional generative adversarial nets (2014) — link
  116. Conditional injective flows for bayesian imaging (2022) — link
  117. Conditional generative adversarial nets for convolutional face generation (2014)
  118. Mixture density networks (1994)
  119. Attention is all you need (2017)
  120. Model projection: Theory and applications to fair machine learning (2020)
  121. Some applications of stochastic differential equations to optimal nonlinear filtering (1964)
  122. Regime changes and financial markets (2012)
  123. Switching Markov Gaussian models for dynamic power system inertia estimation (2015)
  124. Signature moments to characterize laws of stochastic processes (2022)
  125. The identification of nonlinear biological systems: Volterra kernel approaches (1996)
  126. Reservoir kernels and Volterra series (2022) — link
  127. Multilayer feedforward networks are universal approximators (1989)
  128. On the application of discrete-time Volterra series for the damage detection problem in initially nonlinear systems (2017)
  129. Neural sdes as infinite-dimensional gans (2021)
  130. Information projections revisited (2003)
  131. On log-sum inequalities (2023)
  132. Conditioning as disintegration (1997)
  133. Incomplete and asymmetric information in asset pricing theory (2004)
  134. Continual learning with hypernetworks (2019) — link
  135. Convolutional sequence to sequence learning (2017)
  136. Layer normalization (2016) — link
  137. Introduction to the numerical analysis of stochastic delay differential equations (2000)
  138. 13 Stochastic Optimal Control of Delay Equations Arising in Advertising Models (2005)
  139. Stochastic equations with delay: Optimal control via BSDEs and regular solutions of Hamilton–Jacobi–Bellman equations (2010)
  140. Infinite-dimensional Black-Scholes equation with hereditary structure (2007)
  141. A stochastic control problem with delay arising in a pension fund model (2011)

1 of 149 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.