Paper: arXiv 2412.11575

Abstract

This paper considers the finite horizon portfolio rebalancing problem in terms of mean-variance optimization, where decisions are made based on current information on asset returns and transaction costs. The study’s novelty is that the transaction costs are integrated within the optimization problem in a high-dimensional portfolio setting where the number of assets is larger than the sample size. We propose portfolio construction and rebalancing models with nonconvex penalty considering two types of transaction cost, the proportional transaction cost and the quadratic transaction cost. We establish the desired theoretical properties under mild regularity conditions. Monte Carlo simulations and empirical studies using S&P 500 and Russell 2000 stocks show the satisfactory performance of the proposed portfolio and highlight the importance of involving the transaction costs when rebalancing a portfolio.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper involves advanced high-dimensional statistical theory, nonconvex optimization, and convergence proofs, but also demonstrates practical performance through Monte Carlo simulations and empirical backtests on S&P 500 and Russell 2000 data, balancing theoretical depth with empirical validation.

Research Flowchart

  flowchart TD
  A["Research Goal: How to perform cost-aware portfolio rebalancing with nonconvex penalties in a high-dimensional setting (assets >> samples)?"] --> B["Methodology: Mean-Variance Optimization with Integrated Transaction Costs"]
  B --> C["Inputs: S&P 500 & Russell 2000 Data + Monte Carlo Simulations"]
  C --> D["Computational Process: Proportional & Quadratic Cost Modeling with High-Dimensional Estimation"]
  D --> E["Key Findings: Proposed portfolios outperform benchmarks; transaction costs are critical for rebalancing in large universes."]

Cited References (272)

  1. Portfolio optimization with linear and fixed transaction costs (2007)
  2. Markowitz portfolios under transaction costs (2025)
  3. ASYMPTOTIC NORMALITY AND OPTIMALITIES IN ESTIMATION OF LARGE GAUSSIAN GRAPHICAL MODELS (2015)
  4. INNOVATED SCALABLE EFFICIENT ESTIMATION IN ULTRA-LARGE GAUSSIAN GRAPHICAL MODELS (2016)
  5. Tuning-free heterogeneous inference in massive networks (2019)
  6. A direct approach to sparse discriminant analysis in ultra-high dimensions (2012)
  7. The sharpe ratio (1994)
  8. Mutual Fund Performance (1966)
and 264 more
  1. Large Dynamic Covariance Matrices (2019)
  2. Ultra high frequency volatility estimation with dependent microstructure noise (2011)
  3. Efficient estimation of high-dimensional dynamic covariance by risk factor mapping: Applications for financial risk management (2022)
  4. Ultrahigh dimensional precision matrix estimation via refitted cross validation (2020)
  5. Positive semidefinite integrated covariance estimation, factorizations and asynchronicity (2017)
  6. Dominating estimators for minimum-variance portfolios (2010)
  7. High-dimensional minimum variance portfolio estimation based on high-frequency data (2020)
  8. High dimensional minimum variance portfolio estimation under statistical factor models (2021)
  9. Estimation of a multiplicative correlation structure in the large dimensional case (2020)
  10. A nonparametric eigenvalue-regularized integrated covariance matrix estimator for asset return data (2018)
  11. Risks of large portfolios (2015)
  12. Frozen robust multiarray analysis (fRMA) (2010) — link
  13. Variable selection using MM algorithms (2005)
  14. Communication-efficient Distributed Sparse Linear Discriminant Analysis (2017)
  15. Optimal Distributed Online Prediction Using Mini-Batches. (2012)
  16. Model selection consistency of Dantzig selector (2013)
  17. Parallelized Stochastic Gradient Descent. (2010)
  18. Distributed estimation and inference with statistical guarantees (2015) — link
  19. Communication-efficient sparse regression (2017)
  20. On the optimality of averaging in distributed statistical learning (2016)
  21. Communication-efficient algorithms for statistical optimization (2013)
  22. Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates (2015)
  23. Distributed learning, communication complexity and privacy (2012)
  24. Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models (2009)
  25. High-dimensional classification using features annealed independence rules (2008)
  26. Some theory for Fisher’s linear discriminant function, `naive Bayes’, and some alternatives when there are many more variables than observations (2004)
  27. Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties (2001)
  28. NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY (2010)
  29. Bolasso: model consistent lasso estimation through the bootstrap (2008)
  30. A unified framework for high-dimensional analysis of $ M $-estimators with decomposable regularizers (2012)
  31. Thresholding procedures for high dimensional variable selection and statistical estimation (2009)
  32. Approximate sparsity pattern recovery: Information-theoretic lower bounds (2013)
  33. Dimension reduction in regression without matrix inversion (2007)
  34. An iterative coordinate descent algorithm for high-dimensional nonconvex penalized quantile regression (2015)
  35. Multitask quantile regression under the transnormal model (2016)
  36. An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss (2020)
  37. ADMM for penalized quantile regression in big data (2017)
  38. Regularization paths for generalized linear models via coordinate descent (2010)
  39. Pathwise coordinate optimization (2007)
  40. A parallel algorithm for large-scale nonconvex penalized quantile regression (2017)
  41. An augmented ADMM algorithm with application to the generalized lasso problem (2017)
  42. Alternating direction methods for latent variable Gaussian graphical model selection (2013)
  43. Positive-definite $\ell_1$-penalized estimation of large covariance matrices (2012)
  44. Statistical challenges with high dimensionality: Feature selection in knowledge discovery (2006)
  45. An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis (2020)
  46. Variable selection via nonconcave penalized likelihood and its oracle properties (2001)
  47. Sure independence screening for ultrahigh dimensional feature space (2008)
  48. Ultrahigh dimensional feature selection: beyond the linear model (2009)
  49. Sure independence screening in generalized linear models with NP-dimensionality (2010)
  50. Regression shrinkage and selection via the lasso (1996)
  51. Forward regression for ultra-high dimensional variable screening (2009)
  52. On monotropic piecewise quadratic programming (network, algorithm, convex programming, decomposition method) (1986)
  53. Optimal feature selection in high-dimensional discriminant analysis (2014)
  54. Sign consistency for the linear programming discriminant rule (2020)
  55. Linear rate convergence of the alternating direction method of multipliers for convex composite programming (2018)
  56. ET-lasso: a new efficient tuning of lasso-type regularization for high-dimensional data (2019)
  57. The sparse MLE for ultrahigh-dimensional feature screening (2014)
  58. Coordinate descent algorithms for lasso penalized regression (2008)
  59. Efficient distributed learning with sparsity (2017)
  60. Model-free feature screening for ultrahigh-dimensional data (2011)
  61. Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _ 1 $ -Constrained Quadratic Programming (Lasso) (2009)
  62. Nonconcave Penalized Likelihood With NP-Dimensionality (2011)
  63. Strong rules for discarding predictors in lasso-type problems (2012)
  64. Strong Rules for Nonconvex Penalties and Their Implications for Efficient Algorithms in High-Dimensional Regression (2015)
  65. Using generalized correlation to effect variable selection in very high dimensional problems (2009)
  66. Recovery of exact sparse representations in the presence of bounded noise (2005)
  67. Just relax: convex programming methods for identifying sparse signals in noise (2006)
  68. A selective overview of feature screening for ultrahigh-dimensional data (2015)
  69. Tuning variable selection procedures by adding noise (2006)
  70. The lasso, persistence, and cross-validation (2013)
  71. Linear model selection by cross-validation (1993)
  72. Risk consistency of cross-validation with lasso-type procedures (2017)
  73. Controlling variable selection by the addition of pseudovariables (2007)
  74. Controlling the false discovery rate via knockoffs (2015)
  75. Better subset regression using the nonnegative garrote (1995)
  76. An error bound for l1-norm support vector machine coefficients in ultra-high dimension (2016)
  77. The Dantzig selector: statistical estimation when p is much larger than n (2007)
  78. Feature screening via distance correlation learning (2012)
  79. A knockoff filter for high-dimensional selective inference (2016) — link
  80. A selective overview of variable selection in high dimensional feature space (2010)
  81. The adaptive lasso and its oracle properties (2006)
  82. Simultaneous analysis of Lasso and Dantzig selector (2009)
  83. Feature selection for varying coefficient models with ultrahigh-dimensional covariates (2014)
  84. High-dimensional data analysis: The curses and blessings of dimensionality (2000)
  85. Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection (2011)
  86. Feature Screening in Ultrahigh Dimensional Cox’s Model (2016)
  87. Nonparametric Independence Screening in Sparse Ultra-High-Dimensional Additive Models (2011)
  88. Nonparametric independence screening in sparse ultra-high-dimensional varying coefficient models (2014)
  89. Interaction screening for ultrahigh-dimensional data (2014)
  90. Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data (2013)
  91. Feature Screening for Ultrahigh Dimensional Categorical Data With Applications (2014)
  92. The Kolmogorov filter for variable screening in high-dimensional binary classification (2012)
  93. On varying-coefficient independence screening for high-dimensional varying-coefficient models (2014)
  94. TFRC and ACTB as the best reference genes to quantify Urokinase Plasminogen Activator in breast cancer (2011)
  95. Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems (2009)
  96. Gradient methods for minimizing composite functions (2013)
  97. On the linear convergence of descent methods for convex essentially smooth minimization (1992)
  98. Stochastic methods for $\ell_1$-regularized loss minimization (2011)
  99. A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems (2013)
  100. Recovering sparse signals with a certain family of nonconvex penalties and DC programming (2009)
  101. The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems (2005)
  102. HONOR: Hybrid Optimization for Non-convex regularized problems (2015)
  103. Sparsenet: Coordinate descent with nonconvex penalties (2011)
  104. Dc proximal newton for nonconvex optimization problems (2016)
  105. Distributed optimization and statistical learning via the alternating direction method of multipliers (2011)
  106. A new look at the statistical model identification (1974)
  107. Estimating the dimension of a model (1978)
  108. Cross-validation and multinomial prediction (1974)
  109. Tuning parameter selectors for the smoothly clipped absolute deviation method (2007)
  110. Shrinkage tuning parameter selection with a diverging number of parameters (2009)
  111. Estimation stability with cross-validation (ESCV) (2016)
  112. Modified cross-validation for penalized high-dimensional linear regression models (2014)
  113. On Model Selection Consistency of Lasso (2006)
  114. Panning for gold:‘model-X’knockoffs for high dimensional controlled variable selection (2018)
  115. Lasso-type recovery of sparse representations for high-dimensional data (2009)
  116. DECOrrelated feature space partitioning for distributed sparse regression (2016)
  117. A bahadur representation of the linear support vector machine (2008)
  118. Optimally linearizing the alternating direction method of multipliers for convex programming (2020)
  119. Variable selection for support vector machines in moderately high dimensions (2016)
  120. Limiting distributions for $L_1$ regression estimators under general conditions (1998)
  121. Quantile regression for analyzing heterogeneity in ultra-high dimension (2012)
  122. The linearized alternating direction method of multipliers for dantzig selector (2012)
  123. Fast multiple-splitting algorithms for convex optimization (2012)
  124. Fast alternating linearization methods for minimizing the sum of two convex functions (2013)
  125. A dual algorithm for the solution of nonlinear variational problems via finite element approximation (1976)
  126. Sur l’approximation, par 'e l 'e ments finis d’ordre un, et la r 'e solution, par p 'e nalisation-dualit 'e d’une classe de probl `e mes de Dirichlet non lin 'e aires (1975)
  127. DASSO: connections between the Dantzig selector and lasso (2009)
  128. On the linear convergence of the alternating direction method of multipliers (2012)
  129. The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent (2016)
  130. Parallel multi-block ADMM with O (1/k) convergence (2014)
  131. On the sublinear convergence rate of multi-block ADMM (2015)
  132. Alternating direction method with Gaussian back substitution for separable convex programming (2012)
  133. On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming (2015)
  134. Convergence rate and iteration complexity on the alternating direction method of multipliers with a substitution procedure for separable convex programming (2012)
  135. A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints (2015)
  136. Convergence of a block coordinate descent method for nondifferentiable minimization (2001)
  137. Hankel matrix rank minimization with applications to system identification and realization (2013)
  138. Augmented Lagrangian methods: applications to the numerical solution of boundary-value problems (2000)
  139. Convex analysis (2015)
  140. On the global and linear convergence of the generalized alternating direction method of multipliers (2016)
  141. Reappraising medfly longevity: a quantile regression survival analysis (2001)
  142. Hierarchical spline models for conditional quantiles and the demand for electricity (1992)
  143. L1-norm quantile regression (2008)
  144. l1-penalized quantile regression in high-dimensional sparse models (2011)
  145. A direct estimation approach to sparse linear discriminant analysis (2011)
  146. rqPen: Penalized Quantile Regression (2017)
  147. lpSolve: Interface to ‘Lp_solve’ v. 5.5 to Solve Linear/Integer Programs (2015)
  148. A split-and-merge Bayesian variable selection approach for ultrahigh dimensional regression (2015)
  149. Median selection subset aggregation for parallel inference (2014)
  150. On the numerical solution of heat conduction problems in two and three space variables (1956)
  151. An alternating direction method for linear programming (1990)
  152. ADMM for high-dimensional sparse penalized quantile regression (2018)
  153. Quantile regression: 40 years on (2017)
  154. Communication-efficient distributed statistical inference (2018)
  155. Variational analysis (2009)
  156. Parallelism, uniqueness, and large-sample asymptotics for the Dantzig selector (2013)
  157. On the global linear convergence of the admm with multiblock variables (2015)
  158. 1-norm support vector machines (2004)
  159. Support vector machine (1995)
  160. A consistent information criterion for support vector machines in diverging model spaces (2016)
  161. lpsolve: Open source (mixed-integer) linear programming system (2004)
  162. Regression quantiles (1978)
  163. ℓ1-penalized quantile regression in high-dimensional sparse models (2011)
  164. Convex optimization, shape constraints, compound decisions, and empirical Bayes rules (2014)
  165. A convergent 3-block semi-proximal ADMM for convex minimization problems with one strongly convex block (2015)
  166. A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions (2016)
  167. A Two-Phase Augmented Lagrangian Method For Convex Composite Quadratic Programming (2015)
  168. Uncertainty in deep learning (2016)
  169. Backpropagation applied to handwritten zip code recognition (1989)
  170. A deep neural network compression pipeline: Pruning, quantization, huffman encoding (2015) — link
  171. Pruning filters for efficient convnets (2016) — link
  172. Thinet: A filter level pruning method for deep neural network compression (2017) — link
  173. Learning structured sparsity in deep neural networks (2016)
  174. Imagenet classification with deep convolutional neural networks (2012)
  175. Natural language processing (almost) from scratch (2011)
  176. Gradient-based learning applied to document recognition (1998)
  177. Recurrent neural network regularization (2014) — link
  178. Long short-term memory (1997)
  179. Backpropagation through time: what it does and how to do it (1990)
  180. Result Analysis of the NIPS 2003 Feature Selection Challenge (2005)
  181. Calibrating non-convex penalized regression in ultra-high dimension (2013)
  182. Sparse linear discriminant analysis by thresholding for high dimensional data (2011)
  183. Strong oracle optimality of folded concave penalized estimation (2014)
  184. Strong rules for discarding predictors in lasso-type problems (2012)
  185. Strong rules for nonconvex penalties and their implications for efficient algorithms in high-dimensional regression (2015)
  186. Using generalized correlation to effect variable selection in very high dimensional problems (2009)
  187. A selective overview of feature screening for ultrahigh-dimensional data (2015)
  188. Gene expression correlates of clinical prostate cancer behavior (2002)
  189. Nonparametric independence screening in sparse ultra-high-dimensional additive models (2011)
  190. Statistical learning with sparsity: the lasso and generalizations (2015)
  191. Deep feature selection: Theory and application to identify enhancers and promoters (2015)
  192. Simultaneous safe screening of features and samples in doubly sparse modeling (2016)
  193. Deep neural networks for high dimension, low sample size data (2017)
  194. Feature selection using deep neural networks (2015)
  195. Suppressed for Anonymity (2018)
  196. Deep sparse rectifier neural networks (2011)
  197. Regularization and variable selection via the elastic net (2005)
  198. Deep Learning (2016)
  199. Dropout: a simple way to prevent neural networks from overfitting (2014)
  200. Reducing the dimensionality of data with neural networks (2006)
  201. Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015) — link
  202. On statistics, computation and scalability (2013)
  203. Challenges of big data analysis (2014)
  204. A note on the alternating direction method of multipliers (2012)
  205. Optimal Distributed Learning with Multi-pass Stochastic Gradient Methods (2018)
  206. Distributed learning with regularized least squares (2017)
  207. Communication-efficient algorithms for statistical optimization (2012)
  208. FIFA 19 complete player dataset (2019)
  209. Nonconcave penalized likelihood with NP-dimensionality (2011)
  210. One-step sparse estimates in nonconcave penalized likelihood models (2008)
  211. Model selection via Bayesian information criterion for quantile regression models (2014)
  212. Linear rate convergence of the alternating direction method of multipliers for convex composite programming (2017)
  213. Portfolio Selection (1952)
  214. The effect of errors in means, variances, and covariances on optimal portfolio choice (2013)
  215. The Markowitz optimization enigma: Is ‘optimized’optimal? (1989)
  216. Honey, I shrunk the sample covariance matrix (2004)
  217. Covariance regularization by thresholding (2008)
  218. Regularized estimation of large covariance matrices (2008)
  219. Sparsistency and rates of convergence in large covariance matrix estimation (2009)
  220. High dimensional covariance matrix estimation using a factor model (2008)
  221. High dimensional covariance matrix estimation in approximate factor models (2011)
  222. Robust covariance estimation for approximate factor models (2019)
  223. Sparse inverse covariance estimation with the graphical lasso (2008)
  224. High-dimensional Precision Matrix Estimation with a Known Graphical Structure (2021)
  225. Risk reduction in large portfolios: Why imposing the wrong constraints helps (2003)
  226. A generalized approach to portfolio optimization: Improving performance by constraining portfolio norms (2009)
  227. Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? (2009)
  228. Vast portfolio selection with gross-exposure constraints (2012)
  229. Least angle regression (2004)
  230. A linear programming model for selection of sparse high-dimensional multiperiod portfolios (2019)
  231. Convex optimization (2004)
  232. An inexact accelerated stochastic ADMM for separable convex optimization (2022)
  233. Accelerated symmetric ADMM and its applications in signal processing (2019) — link
  234. High dimensional covariancematrix estimation using a factor model (2008)
  235. Improved error rates for sparse (group) learning with Lipschitz loss functions (2021) — link
  236. High-Dimensional Statistics: A Non-Asymptotic Viewpoint (2019)
  237. Theory of financial decision making (1987)
  238. Regularized M-estimators with Nonconvexity: Statistical and Algorithmic Theory for Local Optima (2015)
  239. Projection test for mean vector in high dimensions (2024)
  240. LIFETIME PORTFOLIO SELECTION BY DYNAMIC STOCHASTIC PROGRAMMING (1975)
  241. Predicting with Proxies: Transfer Learning in High Dimension (2020)
  242. Large-scale portfolio allocation under transaction costs and model uncertainty (2019)
  243. High-Dimensional Portfolio Selection with Cardinality Constraints (2022)
  244. Transfer Learning for High-Dimensional Linear Regression: Prediction, Estimation and Minimax Optimality (2021)
  245. A robust perspective on transaction costs in portfolio optimization (2018)
  246. Support recovery without incoherence: A case for nonconvex regularization (2017)
  247. Central limit theorems for high dimensional dependent data (2024)
  248. Sharpe Ratio analysis in high dimensions: Residual-based nodewise regression in factor models (2023)
  249. Approaching Mean-Variance Efficiency for Large Portfolios (2018)
  250. Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _ 1 $ -Constrained Quadratic Programming (Lasso) (2009)
  251. Analytical nonlinear shrinkage of large-dimensional covariance matrices (2020)
  252. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics (2005)
  253. Nonlinear shrinkage of the covariance matrix for portfolio selection: Markowitz meets Goldilocks (2017)
  254. High-dimensional probability: An introduction with applications in data science (2018)
  255. Asset splitting algorithm for ultrahigh dimensional portfolio selection and its theoretical property (2024)
  256. Sparse approximate solutions to linear systems (1995)
  257. A well-conditioned estimator for large-dimensional covariance matrices (2004)
  258. Trading costs and returns for US equities: Estimating effective costs from daily data (2009)
  259. Pure momentum (2025)
  260. An algorithm for portfolio optimization with transaction costs (2005)
  261. Dynamic portfolio optimization with transaction costs: Heuristics and dual bounds (2011)
  262. Large-scale portfolio optimization (1984)
  263. Machine learning and portfolio optimization (2018)
  264. Portfolio Optimization Based on Almost Second-Degree Stochastic Dominance (2024)

0 of 272 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.