Paper: arXiv 2510.14985

Authors: Jinkyu Kim, Hyunjung Yi, Mogan Gim, Donghee Choi, Jaewoo Kang

Abstract

We propose DeepAries , a novel deep reinforcement learning framework for dynamic portfolio management that jointly optimizes the timing and allocation of rebalancing decisions. Unlike prior reinforcement learning methods that employ fixed rebalancing intervals regardless of market conditions, DeepAries adaptively selects optimal rebalancing intervals along with portfolio weights to reduce unnecessary transaction costs and maximize risk-adjusted returns. Our framework integrates a Transformer-based state encoder, which effectively captures complex long-term market dependencies, with Proximal Policy Optimization (PPO) to generate simultaneous discrete (rebalancing intervals) and continuous (asset allocations) actions. Extensive experiments on multiple real-world financial markets demonstrate that DeepAries significantly outperforms traditional fixed-frequency and full-rebalancing strategies in terms of risk-adjusted returns, transaction costs, and drawdowns. Additionally, we provide a live demo of DeepAries at https://deep-aries.github.io/, along with the source code and dataset at https://github.com/dmis-lab/DeepAries, illustrating DeepAries’ capability to produce interpretable rebalancing and allocation decisions aligned with shifting market regimes. Overall, DeepAries introduces an innovative paradigm for adaptive and practical portfolio management by integrating both timing and allocation into a unified decision-making process.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts like Transformer-based state encoders and PPO, warranting a moderate-high math score, while the provision of a live demo, GitHub repository, and extensive backtesting across multiple markets demonstrates significant empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: Adaptively Optimize<br>Portfolio Rebalancing Intervals & Allocations"] --> B["Methodology: DeepAries Framework"]
  B --> C["Input: Multi-Asset Market Data"]
  B --> D["Core: Transformer Encoder<br>Captures Market Dependencies"]
  D --> E["Decision: PPO Agent<br>Outputs Interval (Discrete) & Allocation (Continuous)"]
  E --> F["Execution: Calculate Returns<br>and Transaction Costs"]
  F --> G["Outcome: Adaptive Strategy<br>vs. Baselines (Fixed-Freq/Full-Rebal)"]
  G --> H{"Key Findings"}
  H --> I["Higher Risk-Adjusted Returns"]
  H --> J["Reduced Transaction Costs"]
  H --> K["Improved Drawdowns"]

Cited References (177)

  1. HADAPS: Hierarchical Adaptive Multi-Asset Portfolio Selection (2023) — link
  2. Stop-loss adjusted labels for machine learning-based trading of risky assets (2023) — link
  3. A taxonomy of literature reviews and experimental study of deep reinforcement learning in portfolio management (2025) — link
  4. Can Deep Reinforcement Learning Solve the Portfolio Allocation Problem? (2023) — link
  5. Trend-Heuristic Reinforcement Learning Framework for News-Oriented Stock Portfolio Management (2024)
  6. FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2020) — link
  7. Curriculum learning empowered reinforcement learning for graph-based portfolio management: Performance optimization and comprehensive analysis (2024)
  8. Cross-Insight Trader: A Trading Approach Integrating Policies with Diverse Investment Horizons for Portfolio Management (2024)
and 169 more
  1. Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools — → 2311.10801
  2. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting (2021)
  3. Attention Is All You Need (2017)
  4. Temporal convolutional networks: A unified approach to action segmentation (2016)
  5. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) — link
  6. Opportunistic Rebalancing: A New Paradigm for Wealth Managers (2008)
  7. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting (2023)
  8. Time Series Analysis: Forecasting and Control (1970)
  9. Learning to trade via direct reinforcement (2001)
  10. Long Short-Term Memory (1997)
  11. Reformer: The Efficient Transformer (2020) — link
  12. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling (2014) — link
  13. A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction (2017) — link
  14. Equity2Vec: End-to-end Deep Learning Framework for Cross-sectional Asset Pricing (2021) — link
  15. Stockformer: A Price-Volume Factor Stock Selection Model Based on Wavelet Transform and Multi-Task Self-Attention Networks (2024) — link
  16. MASTER: Market-Guided Stock Transformer for Stock Price Forecasting (2023) — link
  17. Performance Functions and Reinforcement Learning for Trading Systems and Portfolios (1998)
  18. Enhancing Time Series Momentum Strategies Using Deep Neural Networks (2020) — link
  19. Proximal Policy Optimization Algorithms (2017)
  20. Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning (2020) — link
  21. Further Evidence on Investor Overreaction and Stock Market Seasonality (1987)
  22. Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency (1993)
  23. Portfolio Selection (1952)
  24. Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk (1964)
  25. Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting (2021)
  26. Optimistic Bull or Pessimistic Bear: Adaptive Deep Reinforcement Learning for Stock Portfolio Allocation (2019) — link
  27. Relation-Aware Transformer for Portfolio Policy Learning (2020) — link
  28. High-dimensional multi-period portfolio allocation using deep reinforcement learning (2025) — link
  29. Practical Deep Reinforcement Learning Approach for Stock Trading (2018) — link
  30. Commission Fee is Not Enough: A Hierarchical Reinforced Framework for Portfolio Management (2021)
  31. Cost-sensitive portfolio selection via deep reinforcement learning (2022) — link
  32. FreQuant: A Reinforcement-Learning based Adaptive Portfolio Optimization with Multi-frequency Decomposition (2024)
  33. WaveCorr: Deep reinforcement learning with permutation invariant convolutional policy networks for portfolio management (2023) — link
  34. Mastering Stock Markets with Efficient Mixture of Diversified Trading Experts (2023) — link
  35. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (2023) — link
  36. Transformer Conformal Prediction for Time Series (2024) — link
  37. A Joint Time-Frequency Domain Transformer for Multivariate Time Series Forecasting (2023) — link
  38. A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case (2023) — link
  39. Forecasting: Principles and Practice (2008)
  40. Support Vector Regression Machines (1997)
  41. Learning Representations by Back-propagating Errors (1986)
  42. Portfolio Transformer for Attention-Based Asset Allocation (2022) — link
  43. News Sentiment and Financial Market Prediction: A Transformer-Based Approach (2021)
  44. Q-learning (1992)
  45. Human-level Control through Deep Reinforcement Learning (2015)
  46. Continuous Control with Deep Reinforcement Learning (2015) — link
  47. Learning to Summarize with Human Feedback (2020)
  48. Llama 2: Open Foundation and Fine-Tuned Chat Models (2023) — link
  49. Predicting Stock Closing Prices in Emerging Markets with Transformer Neural Networks: The Saudi Stock Exchange Case (2021)
  50. Transformer-Based Deep Learning Model for Stock Price Prediction: A Case Study on Bangladesh Stock Market (2023)
  51. Multi-Head Self-Attention Transformer for Dogecoin Price Prediction (2021)
  52. Stock Market Index Prediction Using Deep Transformer Model (2022)
  53. Stock Price Prediction Based on Temporal Fusion Transformer (2021)
  54. Training Language Models to Follow Instructions with Human Feedback (2022)
  55. Aligning Large Multimodal Models with Factually Augmented RLHF (2023) — link
  56. Tallrec: An Effective and Efficient Tuning Framework to Align Large Language Models with Recommendation (2023)
  57. One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds Without Per-Shape Optimization (2024)
  58. Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization (2023) — link
  59. Few-shot Parameter-Efficient Fine-Tuning Is Better and Cheaper than In-Context Learning (2022)
  60. The Power of Scale for Parameter-Efficient Prompt Tuning (2021) — link
  61. Bi-LORA: A Vision-Language Approach for Synthetic Image Detection (2024) — link
  62. Deep Direct Reinforcement Learning for Financial Signal Representation and Trading (2016)
  63. PEFT: State-of-the-art Parameter-Efficient Fine-Tuning Methods (2022)
  64. Reinforcement Learning: An Introduction (2018)
  65. Asynchronous Methods for Deep Reinforcement Learning (2016)
  66. Trust Region Policy Optimization (2015)
  67. Prolificdreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (2024)
  68. Llama-adapter: Efficient Fine-Tuning of Language Models with Zero-Init Attention (2023) — link
  69. GitHub Repository for Alpaca-lora (2024) — link
  70. T2i-adapter: Learning Adapters to Dig Out More Controllable Ability for Text-to-Image Diffusion Models (2024)
  71. mplug-owl: Modularization Empowers Large Language Models with Multimodality (2023) — link
  72. TLT: Recurrent Fine-Tuning Transfer Learning for Water Quality Long-Term Prediction (2022)
  73. Spatio-Temporal Meta-Graph Learning for Traffic Forecasting (2023)
  74. Flight Trajectory Prediction Enabled by Time-Frequency Wavelet Transform (2023)
  75. Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals (2011)
  76. Style Investing: Unique Insight into Equity Management (1995)
  77. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018)
  78. Nonlinear Dynamics and Stock Returns (1989)
  79. Mean Reversion in Stock Prices: Evidence and Implications (1988)
  80. Mean-Reverting Portfolio with Budget Constraint (2018)
  81. Estimated Correlation Matrices and Portfolio Optimization (2004)
  82. Forecasting Water Use in Selected Water Supply Systems (2017)
  83. Long-Term Electricity Forecast: A Systematic Review (2015)
  84. Time Series Analysis and Forecasting on Crime Data (2022)
  85. Generative Adversarial Networks for Biomedical Time Series Forecasting and Imputation (2022)
  86. Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting (2021)
  87. Time-Series Generative Adversarial Networks (2019)
  88. TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks (2020)
  89. Parameter-Efficient Transfer Learning for NLP (2019)
  90. Lora: Low-Rank Adaptation of Large Language Models (2021) — link
  91. Machine Learning Techniques and Data for Stock Market Forecasting: A Literature Review (2022)
  92. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021) — link
  93. Portfolio Selection Based on Upper and Lower Exponential Possibility Distributions (1999)
  94. A Survey of Credibility Theory (2006)
  95. Exponential Gradient with Momentum for Online Portfolio Selection (2022)
  96. A Novel Deep Learning Framework: Prediction and Analysis of Financial Time Series Using CEEMD and LSTM (2020)
  97. Portfolio Formation with Preselection Using Deep Learning from Long-Term Financial Data (2020)
  98. Stock Price Prediction Using Deep Learning and Frequency Decomposition (2021)
  99. Forecasting Trends in Time Series (1985)
  100. A New Approach to Portfolio Selection Based on Forecasting (2023)
  101. Portfolio Selection with Skewness: A Multiple-Objective Approach (1991)
  102. Mean-Absolute Deviation Portfolio Optimization Model and Its Applications to Tokyo Stock Market (1991)
  103. Optimization of Conditional Value-at-Risk (2000)
  104. Fedformer: Frequency Enhanced Decomposed Transformer for Long-Term Series Forecasting (2022)
  105. HADAPS: Hierarchical Adaptive Multi-Asset Portfolio Selection (2023)
  106. MAPS: Multi-Agent Reinforcement Learning-Based Portfolio Management System (2021)
  107. Playing Atari with Deep Reinforcement Learning (2013) — link
  108. Reinforcement Learning with Deep Energy-Based Policies (2017)
  109. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018)
  110. Soft Actor-Critic Algorithms and Applications (2018) — link
  111. Goal-Conditioned Reinforcement Learning with Imagined Subgoals (2021)
  112. Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies (2021)
  113. A Universal Performance Measure (2002)
  114. Listening to Chaotic Whispers: A Deep Learning Framework for News-Oriented Stock Trend Prediction (2018)
  115. Stock Movement Prediction from Tweets and Historical Prices (2018)
  116. Technical Analysis and Sentiment Embeddings for Market Trend Prediction (2019)
  117. AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy Using Interpretable Deep Reinforcement Attention Networks (2019)
  118. Bitcoin Volatility Forecasting with a Glimpse into Buy and Sell Orders (2018)
  119. A Deep Q-Learning Portfolio Management Framework for the Cryptocurrency Market (2020)
  120. An Intelligent Financial Portfolio Trading Strategy Using Deep Q-Learning (2020)
  121. Deep Reinforcement Learning Based Trading Agents: Risk Curiosity Driven Learning for Financial Rules-Based Policy (2021)
  122. Risk and Asset Allocation (2005)
  123. Markowitz’s Mean-Variance Portfolio Selection with Regime Switching: A Continuous-Time Model (2003)
  124. Stock Market Volatility and Learning (2016)
  125. Portfolio Selection Using Neural Networks (2007)
  126. Ensemble of Technical Analysis and Machine Learning for Market Trend Prediction (2018)
  127. Artificial Neural Networks Architectures for Stock Price Prediction: Comparisons and Applications (2016)
  128. Hierarchical Portfolio Management: Theory and Applications (2007)
  129. Long-Term Global Market Correlations (2001)
  130. International Diversification of Investment Portfolios (1970)
  131. Is the Correlation in International Equity Returns Constant: 1960–1990? (1995)
  132. Why Not Diversify Internationally Rather Than Domestically? (1974)
  133. Is There a Place for Property in the Multi-Asset Portfolio? (1995)
  134. Toward Maximum Diversification (2008)
  135. Correlation Risk and Optimal Portfolio Choice (2010)
  136. Nonparametric Correlation Models for Portfolio Allocation (2013)
  137. Hierarchical Clustering-Based Asset Allocation (2017)
  138. Estimated Correlation Matrices and Portfolio Optimization (2004)
  139. Effect of Portfolio Diversification Practices on Cash Flow Management among National Technical and Vocational Education and Training Institutions in Uasin Gishu County, Kenya (2020)
  140. Policy Gradient Methods for Reinforcement Learning with Function Approximation (1999)
  141. Benchmarking Deep Reinforcement Learning for Continuous Control (2016)
  142. High-Dimensional Continuous Control Using Generalized Advantage Estimation (2015) — link
  143. An Introduction to Omega (2002)
  144. Beyond Sharpe Ratio: Optimal Asset Allocation Using Different Performance Ratios (2008)
  145. Time Series Momentum (2012)
  146. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
  147. DeepTrader: A Deep Reinforcement Learning Approach for Risk-Return Balanced Portfolio Management with Market Conditions Embedding (2021) — link
  148. Commission Fee is Not Enough: A Hierarchical Reinforced Framework for Portfolio Management (2021)
  149. Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency (1993)
  150. Further Evidence on Investor Overreaction and Stock Market Seasonality (1987)
  151. DeepClair: Utilizing Market Forecasts for Effective Portfolio Selection (2024)
  152. Conditional Time Series Forecasting with Convolutional Neural Networks (2017)
  153. ArkDTA: attention regularization guided by non-covalent interactions for explainable drug–target binding affinity prediction (2023)
  154. Where Is the Value Added of Rebalancing? A Systematic Comparison of Alternative Rebalancing Strategies (2014)
  155. Optimal Dynamic Portfolio Rebalancing (2024)
  156. A Robust Perspective on Transaction Costs in Portfolio Optimization (2018)
  157. An Adaptive Portfolio Trading System: A Risk-Return Portfolio Optimization Using Recurrent Reinforcement Learning with Expected Maximum Drawdown (2017)
  158. Deep Reinforcement Learning for Portfolio Optimization with Transaction Costs and Risk Control (2024)
  159. Transformer-based Deep Reinforcement Learning for High-frequency Portfolio Management (2023)
  160. Adaptive Portfolio Rebalancing via Reinforcement Learning with PPO (2025)
  161. Dynamic Strategies for Asset Allocation (1988)
  162. Scilab — link
  163. Scilab — link
  164. The Computational Geometry Algorithms Library — link
  165. The Computational Geometry Algorithms Library — link
  166. The Parmap library — link
  167. The Parmap library — link
  168. BabyJ : from Object Based to Class Based Programming via Types (2003)
  169. tag-sound

1 of 177 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.