Paper: arXiv 2505.11243

Authors: Elliot L. Epstein, Apaar Sadhwani, Kay Giesecke

Abstract

Many prediction problems across science and engineering, especially in finance and economics, involve large cross-sections of individual time series, where each unit (e.g., a loan, stock, or customer) is driven by unit-level features and latent cross-sectional dynamics. While sequence models have advanced per-unit temporal prediction, capturing cross-sectional effects often still relies on hand-crafted summary features. We propose Set-Sequence, a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering. At each time step, a permutation-invariant Set module summarizes the unit set; a Sequence module then models each unit’s dynamics conditioned on both its features and the learned summary. The architecture accommodates unaligned series, supports varying numbers of units at inference, integrates with standard sequence backbones (e.g., Transformers), and scales linearly in cross-sectional size. Across a synthetic contagion task and two large-scale real-world applications, equity portfolio optimization and loan risk prediction, Set-Sequence significantly outperforms strong baselines, delivering higher Sharpe ratios, improved AUCs, and interpretable cross-sectional summaries.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel neural architecture with theoretical foundations, involving permutation invariance and complexity analysis, which indicates advanced mathematics. It also demonstrates strong empirical results on large-scale, real-world financial datasets (portfolio optimization and mortgage prediction) with quantitative metrics like Sharpe ratios and AUCs, showing high backtest readiness.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Cross-Sectional Time Series Prediction"] --> B["Methodology:<br>Set-Sequence Model"]
  B --> C["Permutation-Invariant Set Module<br>Summarizes Unit Set"]
  C --> D["Sequence Module<br>Models Dynamics per Unit"]
  D --> E["Training & Evaluation"]
  E --> F["Key Outcomes:<br>Better Sharpe/AUC, Interpretability"]
  
  B -->|Inputs| G["Data: Finance/Econ Cross-Sections<br>(e.g., Loans, Stocks)"]
  
  style F fill:#e1f5e1,stroke:#2e7d32,stroke-width:2px

Cited References (68)

  1. Simple hardware-efficient long convolutions for sequence modeling (2023)
  2. Effectively Modeling Time Series with Simple Discrete State Spaces (2023) — link
  3. Efficiently Modeling Long Sequences with Structured State Spaces (2022) — link
  4. Mamba: Linear-Time Sequence Modeling with Selective State Spaces (2024) — link
  5. Is Mamba effective for time series forecasting? (2025) — link
  6. Fredformer: Frequency Debiased Transformer for Time Series Forecasting (2024) — link
  7. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (2024) — link
  8. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting (2023) — link
and 60 more
  1. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021) — link
  2. Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting (2022) — link
  3. FED former: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting (2022) — link
  4. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021) — link
  5. TSM ixer: An All- MLP Architecture for Time Series Forecast-ing (2023) — link
  6. Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023) — link
  7. Exploring the sources of default clustering (2018) — link
  8. Consumer credit-risk models via machine-learning algorithms (2010) — link
  9. Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets (1983)
  10. Macroeconomics and Reality (1980)
  11. Affine Point Processes and Portfolio Credit Risk (2010)
  12. Deep Learning for Mortgage Risk* (2020) — link
  13. Portfolio Transformer for Attention-Based Asset Allocation (2023) — link
  14. Set transformer: A framework for attention-based permutation-invariant neural networks (2019)
  15. Deep Sets (2017) — link
  16. Rep the Set: Neural Networks for Learning Set Representations (2019) — link
  17. DeepSetNet: Predicting Sets with Deep Neural Networks (2016) — link
  18. Joint Learning of Set Cardinality and State Distribution (2017) — link
  19. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021) — link
  20. Set-based Neural Network Encoding (2023) — link
  21. Set functions for time series (2020)
  22. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks (2020) — link
  23. Discrete Graph Structure Learning for Forecasting Multiple Time Series (2021) — link
  24. Graph neural network for traffic forecasting: A survey (2022) — link
  25. Moirai: A Time Series Foundation Model for Universal Forecasting (2024) — link
  26. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series (2022)
  27. Time series as images: Vision transformer for irregularly sampled time series (2024)
  28. Warpformer: A multi-scale modeling approach for irregular clinical time series (2023)
  29. DCSF: deep convolutional set functions for classification of asynchronous time series (2022)
  30. Tripletformer for Probabilistic Interpolation of Irregularly sampled Time Series (2023)
  31. Latent Linear ODE s with Neural Kalman Filtering for Irregular Time Series Forecasting (2023) — link
  32. Time-parameterized convolutional neural networks for irregularly sampled time series (2023) — link
  33. Deep Factors for Forecasting (2019) — link
  34. Macroeconomic Forecasting Using Diffusion Indexes (2002) — link
  35. Dynamic Factor Models (2011) — link
  36. Macroeconomic nowcasting and forecasting with big data (2018)
  37. Forecasting using principal components from a large number of predictors (2002)
  38. Are more data always better for factor analysis? (2006)
  39. A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series (2006) — link
  40. How is machine learning useful for macroeconomic forecasting? (2022)
  41. Large panels with common factors and spatial correlation (2011)
  42. Panel time series (2010)
  43. Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions (2006)
  44. Deep learning in characteristics-sorted factor models (2024)
  45. The Dynamic, the Static, and the Weak factor models and the analysis of high-dimensional time series (2025) — link
  46. High-dimensional multivariate forecasting with low-rank Gaussian Copula Processes (2019) — link
  47. Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting (2019) — link
  48. Normalizing Kalman Filters for Multivariate Time Series Analysis (2020) — link
  49. Statistical Analysis with Missing Data (2014) — link
  50. A New Approach to Linear Filtering and Prediction Problems (1960) — link
  51. Deep Learning in Asset Pricing (2024) — link
  52. Attention is all you need (2017)
  53. Hyena hierarchy: Towards larger convolutional language models (2023)
  54. Hungry hungry hippos: Towards language modeling with state space models (2022) — link
  55. Deep Learning Statistical Arbitrage (2022)
  56. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting (2024) — link
  57. The Graph Neural Network Model (2009) — link
  58. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks (2020) — link
  59. TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (2024) — link
  60. TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis (2025) — link

0 of 68 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.