Reinforcement Learning for Monetary Policy Under Macroeconomic Uncertainty: Analyzing Tabular and Function Approximation Methods

Paper: arXiv 2512.17929

Authors: Tony Wang, Kyle Feinstein, Sheryl Chen

Abstract

We study how a central bank should dynamically set short-term nominal interest rates to stabilize inflation and unemployment when macroeconomic relationships are uncertain and time-varying. We model monetary policy as a sequential decision-making problem where the central bank observes macroeconomic conditions quarterly and chooses interest rate adjustments. Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function. We chose to compare nine different reinforcement learning style approaches against Taylor Rule and naive baselines, including tabular Q-learning variants, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning with uncertainty quantification, and POMDP formulations with partial observability. Notably, despite its simplicity, standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return), outperforming both enhanced RL methods and traditional policy rules. Our results suggest that while sophisticated RL techniques show promise for monetary policy applications, simpler approaches may be more robust in this domain, highlighting important challenges in applying modern RL to macroeconomic policy.

Complexity vs Empirical Score

• Math Complexity: 6.5/10
• Empirical Rigor: 7.2/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces advanced reinforcement learning concepts and formal MDP/POMDP models, indicating moderate-to-high mathematical density. However, it is backed by concrete empirical implementation using real-world FRED data, specific backtesting procedures, and detailed performance metrics with statistical analysis.

Research Flowchart

  flowchart TD
  A["Research Goal: RL for Monetary Policy under Uncertainty"] --> B["Data & Inputs<br>FRED Macroeconomic Data"]
  B --> C["Model: MDP &<br>Linear-Gaussian Transition"]
  C --> D{"RL Methods Comparison"}
  D --> E["Tabular Methods<br>Q-Learning, SARSA"]
  D --> F["Deep Methods<br>DQN, Actor-Critic"]
  D --> G["Uncertainty Methods<br>Bayesian Q-Learning, POMDP"]
  E & F & G --> H["Key Finding: Simplicity Wins<br>Tabular Q-Learning (-615.13) Outperforms Complex Methods"]

Cited References (16)

1. The Courage to Act: A Memoir of a Crisis and Its Aftermath (2015)
2. Monetary Policy Communications and Their Effects on Household Inflation Expectations (2020)
3. Bayesian Reinforcement Learning: A Survey (2015)
4. Planning and Acting in Partially Observable Stochastic Domains (1998)
5. Actor-Critic Algorithms (2000)
6. Human-Level Control through Deep Reinforcement Learning (2015)
7. Identification in Macroeconomics (2018)
8. Deep Exploration via Bootstrapped DQN (2016)

0 of 16 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.