Paper: arXiv 2407.20352

Abstract

This article describes the methods that achieved 4th and 6th place in the forecasting and investment challenges, respectively, of the M6 competition, ultimately securing the 1st place in the overall duathlon ranking. In the forecasting challenge, we tested a novel meta-learning model that utilizes hypernetworks to design a parametric model tailored to a specific family of forecasting tasks. This approach allowed us to leverage similarities observed across individual forecasting tasks while also acknowledging potential heterogeneity in their data generating processes. The model’s training can be directly performed with backpropagation, eliminating the need for reliance on higher-order derivatives and is equivalent to a simultaneous search over the space of parametric functions and their optimal parameter values. The proposed model’s capabilities extend beyond M6, demonstrating superiority over state-of-the-art meta-learning methods in the sinusoidal regression task and outperforming conventional parametric models on time-series from the M4 competition. In the investment challenge, we adjusted portfolio weights to induce greater or smaller correlation between our submission and that of other participants, depending on the current ranking, aiming to maximize the probability of achieving a good rank.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel hypernetwork-based meta-learning architecture with a formal mathematical treatment (e.g., parameterization, loss functions) while being heavily grounded in empirical results from the competitive M6, M4, and sinusoidal regression benchmarks, including code and backtest implementation.

Research Flowchart

  flowchart TD
  A["Research Goal: <br>Design Model for M6 <br>Forecasting & Investment <br>Challenges"] --> B1["Forecasting Challenge <br>Meta-Learning with Hypernetworks"]
  A --> B2["Investment Challenge <br>Game-Theoretic Portfolio"]

  B1 --> C1["Data: M4/M6 Time-Series"]
  B2 --> C2["Data: Participant Rankings"]

  C1 --> D1["Process: Simultaneous search <br>for parametric functions <br>& optimal parameters"]
  C2 --> D2["Process: Adjust weights to <br>induce correlation <br>for rank maximization"]

  D1 --> E1["Outcome: 4th Place<br>Forecasting"]
  D2 --> E2["Outcome: 6th Place<br>Investment"]

  E1 & E2 --> F["Final Outcome:<br>1st Place Overall<br>Duathlon Ranking"]

Cited References (80)

  1. How Important Is the Train-Validation Split in Meta-Learning ? (2021)
  2. Improving the Accuracy of Global Forecasting Models Using Time Series Data Augmentation (2021) — link
  3. Hypernetworks in Meta-Reinforcement Learning (2023)
  4. SMASH : One-Shot Model Architecture Search through HyperNetworks (2017) — link
  5. Forecasting Analogous Time Series (2001) — link
  6. Model- Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017)
  7. Meta- Learning with Warped Gradient Descent (2020) — link
  8. On the Modularity of Hypernetworks (2020)
and 72 more
  1. Ensembles of Localised Models for Time Series Forecasting (2021) — link
  2. HyperNetworks (2016) — link
  3. Global Models for Time Series Forecasting : A Simulation Study (2021) — link
  4. Recurrent Neural Networks for Time Series Forecasting : Current Status and Future Directions (2021) — link
  5. Meta-Learning in Neural Networks: A Survey (2021)
  6. Bayesian Model-Agnostic Meta-Learning (2018) — link
  7. Gradient- Based Meta-Learning with Learned Layerwise Metric and Subspace (2018)
  8. Meta- SGD : Learning to Learn Quickly for Few-Shot Learning (2017) — link
  9. To Learn Effective Features : Understanding the Task-Specific Adaptation of MAML (2020)
  10. Learning to Learn: An Introduction to Meta Learning (2020)
  11. The M5 Competition: Background , Organization, and Implementation (2021) — link
  12. Predicting/Hypothesizing the Findings of the M5 Competition (2021) — link
  13. Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality (2021) — link
  14. Meta- Learning via Classifier (-Free) Diffusion Guidance (2023) — link
  15. Meta-Curvature (2019)
  16. A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction (2017) — link
  17. Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of Maml (2019) — link
  18. Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts Using Meta-Learning Hypernetworks (2023) — link
  19. Early Stopping and Non-Parametric Regression: An Optimal Data-Dependent Stopping Rule (2014)
  20. Hands- On Meta Learning with Python : Meta Learning Using One-Shot Learning, MAML , Reptile , and Meta-SGD with TensorFlow (2018)
  21. Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes (2020) — link
  22. Hands- On Transfer Learning with Python : Implement Advanced Deep Learning and Neural Network Models Using TensorFlow and Keras (2018)
  23. Personalized Federated Learning Using Hypernetworks (2021)
  24. Meta- Learning : A Survey (2018) — link
  25. Continual Learning with Hypernetworks (2022) — link
  26. Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation (2021)
  27. Meta-Learning in Natural and Artificial Intelligence (2021) — link
  28. HyperDynamics : Meta-Learning Object and Agent Dynamics with Hypernetworks (2021) — link
  29. Deep Multi-Task Representation Learning: A Tensor Factorisation Approach (2016) — link
  30. On Early Stopping in Gradient Descent Learning (2007)
  31. Meta- Learning via Hypernetworks (2020) — link
  32. Fast Context Adaptation via Meta-Learning (2019)
  33. Hierarchical Forecasting with a Top-down Alignment of Independent-Level Forecasts (2022) — link
  34. A Fast and Scalable Ensemble of Global Models with Long Memory and Data Partitioning for the M5 Forecasting Competition (2022) — link
  35. Forecasting across Time Series Databases Using Recurrent Neural Networks on Groups of Similar Series: A Clustering Approach (2020) — link
  36. Xgboost: Extreme Gradient Boosting (2023)
  37. Modeling Volatility Dynamics (1995) — link
  38. SETAR-Tree : A Novel and Accurate Tree Algorithm for Global Time Series Forecasting (2023) — link
  39. Risks from Learned Optimization in Advanced Machine Learning Systems (2021) — link
  40. A Survey of Deep Meta-Learning (2021) — link
  41. Simple Averaging of Direct and Recursive Forecasts via Partial Pooling Using Machine Learning (2022) — link
  42. M6 Competition , ISF 2023 Practitioner Talk (2023)
  43. A Combination-Based Forecasting Method for the M4-competition (2020) — link
  44. Criteria for Classifying Forecasting Methods (2020) — link
  45. The Sharpe Ratio of Estimated Efficient Portfolios (2016) — link
  46. Robust Performance Hypothesis Testing with the Sharpe Ratio (2008) — link
  47. The M6 Financial Duathlon Competition Guidelines (2022)
  48. The M6 Forecasting Competition: Bridging the Gap between Forecasting and Investment Decisions (2023) — link
  49. Reflections on the Efficient Market Hypothesis : 30 Years Later (2005) — link
  50. A Method of Simulated Moments for Estimation of Discrete Response Models Without Numerical Integration (1989) — link
  51. FinQBoost : Machine Learning for Portfolio Forecasting (2023)
  52. The Options Market Beat 94% of Participants in the M6 Financial Forecasting Contest (2023)
  53. Weighted Ensemble of Statistical Models (2020) — link
  54. Multivariate Observations (1984)
  55. Investigating the Accuracy of Cross-Learning Time Series Forecasting Methods (2021) — link
  56. Data Preprocessing and Augmentation for Multiple Short Time Series Forecasting with Recurrent Neural Networks (2016)
  57. A Hybrid Method of Exponential Smoothing and Recurrent Neural Networks for Time Series Forecasting (2020) — link
  58. A Note on the M6 Forecasting Competition : Designing Parametric Models with Hypernetworks (2023) — link
  59. A Note on the M6 Forecasting Competition : Rank Optimization (2023) — link
  60. TTR : Technical Trading Rules (2021)
  61. S& P 500 Returns since 1930 (2023)
  62. Implementing Transfer Learning across Different Datasets for Time Series Forecasting (2021) — link
  63. A Survey on Multi-Task Learning (2022) — link
  64. Automatic Time Series Forecasting : The Forecast Package for R (2008) — link
  65. A State Space Framework for Automatic Forecasting Using Exponential Smoothing Methods (2002) — link
  66. Reduced-Rank Regression for the Multivariate Linear Model (1975) — link
  67. Time Series Forecasting with Hypernetworks Generating Parameters in Advance (2022) — link
  68. One Shot Learning of Simple Visual Concepts (2011)
  69. Optimization as a Model for Few-Shot Learning (2016)
  70. The M4 Competition : 100,000 Time Series and 61 Forecasting Methods (2020) — link
  71. Some Cautionary Notes on the Use of Principal Components Regression (1998) — link
  72. Tsfeatures: Time Series Feature Extraction (2023)

0 of 80 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.