AI-Powered Energy Algorithmic Trading: Integrating Hidden Markov Models with Neural Networks

Paper: arXiv 2407.19858

Abstract

In quantitative finance, machine learning methods are essential for alpha generation. This study introduces a new approach that combines Hidden Markov Models (HMM) and neural networks, integrated with Black-Litterman portfolio optimization. During the COVID period (2019-2022), this dual-model approach achieved a 83% return with a Sharpe ratio of 0.77. It incorporates two risk models to enhance risk management, showing efficiency during volatile periods. The methodology was implemented on the QuantConnect platform, which was chosen for its robust framework and experimental reproducibility. The system, which predicts future price movements, includes a three-year warm-up to ensure proper algorithm function. It targets highly liquid, large-cap energy stocks to ensure stable and predictable performance while also considering broker payments. The dual-model alpha system utilizes log returns to select the optimal state based on the historical performance. It combines state predictions with neural network outputs, which are based on historical data, to generate trading signals. This study examined the architecture of the trading system, data pre-processing, training, and performance. The full code and backtesting data are available under the QuantConnect terms: https://github.com/tiagomonteiro0715/AI-Powered-Energy-Algorithmic-Trading-Integrating-Hidden-Markov-Models-with-Neural-Networks

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 8.5/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical frameworks including Hidden Markov Models, neural networks, and Black-Litterman optimization with explicit LaTeX formulas, while demonstrating high empirical rigor through live backtesting on QuantConnect, publication of code, and detailed performance metrics.

Research Flowchart

  flowchart TD
  A["Research Goal: Validate HMM + NN for Energy Trading Alpha"] --> B["Data & Model Selection"]
  B --> C{"Data Pre-Processing & Training"}
  
  subgraph B [" "]
      B1["Input: Energy Stocks<br>2019-2022"]
      B2["Model 1: Hidden Markov Model<br>States + Log Returns"]
      B3["Model 2: Neural Network<br>Price Prediction"]
  end
  
  subgraph C [" "]
      C1["Backtesting on QuantConnect<br>3-Year Warm-up"]
      C2["Alpha Signal Generation"]
      C3["Integration with<br>Black-Litterman Optimization"]
  end
  
  C --> D["Execution & Risk Management"]
  
  subgraph D [" "]
      D1["Trade: Liquidity & Cost Analysis"]
      D2["Outcome: 83% Return<br>Sharpe 0.77"]
  end
  
  D --> E["Final Output: Open Source Strategy"]

Cited References (28)

1. Unleashing the algorithmic frontier: Navigating the impact of algo trading on investor portfolios (2023) — link
2. Algorithmic Trading and AI : A Review of Strategies and Market Impact (2024) — link
3. Detecting bearish and bullish markets in financial time series using hierarchical hidden Markov models (2020) — link
4. Neural Networks (2022) — link
5. Robust machine learning models: linear and nonlinear (2024) — link
6. Pairwise Markov models for stock index forecasting (2017) — link
7. A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN , RNN , LSTM , GRU (2023) — link
8. Explainable Artificial Intelligence based Ensemble Machine Learning for Ovarian Cancer Stratification using Electronic Health Records (2023) — link

0 of 28 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.