Paper: arXiv 2508.20097

Authors: Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

Abstract

We investigate whether large language models can discover and analyze U.S. tax-minimization strategies. This real-world domain challenges even seasoned human experts, and progress can reduce tax revenue lost from well-advised, wealthy taxpayers. We evaluate the most advanced LLMs on their ability to (1) interpret and verify tax strategies, (2) fill in gaps in partially specified strategies, and (3) generate complete, end-to-end strategies from scratch. This domain should be of particular interest to the LLM reasoning community: unlike synthetic challenge problems or scientific reasoning tasks, U.S. tax law involves navigating hundreds of thousands of pages of statutes, case law, and administrative guidance, all updated regularly. Notably, LLM-based reasoning identified an entirely novel tax strategy, highlighting these models’ potential to revolutionize tax agencies’ fight against tax abuse.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper’s core math is low-complexity linguistic prompting and legal analysis, but it demonstrates high empirical rigor through a handcrafted dataset (Shelter Check), GitHub code, and expert-graded strategy generation from real-world LLMs.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Can LLMs Identify Tax Abuse?"] --> B["Methodology: Three Evaluation Tasks<br>(1) Interpret & Verify<br>(2) Fill Gaps<br>(3) Generate Strategies"]
  B --> C["Computational Process<br>LLM Reasoning on Tax Law<br>Simulated Expert Interactions"]
  C --> D["Key Findings & Outcomes<br>Novel Strategy Discovery<br>Verification Capability<br>Potential for Tax Agency Use"]

Cited References (64)

  1. Rethinking the Development of Large Language Models from the Causal Perspective: A Legal Text Prediction Case Study (2024) — link
  2. RR-PU: A Synergistic Two-Stage Positive and Unlabeled Learning Framework for Robust Tax Evasion Detection (2024) — link
  3. GxVAEs: two joint VAEs generate hit molecules from gene expression profiles (2024) — link
  4. Black-Box Optimization with Implicit Constraints for Public Policy (2025) — link
  5. J & H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain (2025) — link
  6. Blackboard Systems (1986)
  7. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education (1983)
  8. Classification Problem Solving (1984)
and 56 more
  1. New Ways to Make Microcircuits Smaller (1980) — link
  2. New Ways to Make Microcircuits Smaller—Duplicate Entry (1980)
  3. Strategic explanations for a diagnostic consultation system (1984) — link
  4. Strategic Explanations in Consultation—Duplicate (1983)
  5. Poligon: A System for Parallel Problem Solving (1986)
  6. Transfer of Rule-Based Expertise through a Tutorial Dialogue (1979)
  7. The Engineering of Qualitative Models (2021)
  8. Attention Is All You Need (2017) — link
  9. Pluto: The ‘Other’ Red Planet (2015)
  10. Deep Blue (2002) — link
  11. A Survey on Large Language Models for Automated Planning (2025) — link
  12. Overview and Discussion of the Competition on Legal Information, Extraction/Entailment (COLIEE) 2023 (2024) — link
  13. Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark (2021) — link
  14. JEC-QA: A Legal-Domain Question Answering Dataset (2020) — link
  15. ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation (2021) — link
  16. LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2022) — link
  17. Computational Discovery of Transaction-Based Financial Crime via Grammatical Evolution: The Case of Ponzi Schemes (2022) — link
  18. Can AI expose tax loopholes? Towards a new generation of legal policy assistants (2025) — link
  19. Attention is All you Need (2017) — link
  20. Accurate structure prediction of biomolecular interactions with AlphaFold 3 (2024)
  21. Machine science (2010)
  22. GPT Gets Its First Law School B-pluses (2024)
  23. Can GPT-3 Perform Statutory Reasoning? (2023) — link
  24. A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering (2020) — link
  25. LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models (2023) — link
  26. Detecting tax evasion: a co-evolutionary approach (2016) — link
  27. Using LLMs to Discover Legal Factors (2024) — link
  28. Predicting Outcomes of Case-Based Legal Arguments (2003) — link
  29. The Design of a Legal Analysis Program (1983) — link
  30. Catala: a programming language for the law (2021) — link
  31. The British Nationality Act as a logic program (1986) — link
  32. Reflections on ‘‘Taxman": An Experiment in Artificial Intelligence and Legal Reasoning (1977) — link
  33. Crafting Papers on Machine Learning (2000)
  34. The Need for Biases in Learning Generalizations (1980)
  35. Computational Complexity of Machine Learning (1989)
  36. Machine Learning: An Artificial Intelligence Approach, Vol. I (1983)
  37. Pattern Classification (2000)
  38. Suppressed for Anonymity (2021)
  39. Mechanisms of Skill Acquisition and the Law of Practice (1981)
  40. Some Studies in Machine Learning Using the Game of Checkers (1959)
  41. Wiz Research Uncovers Exposed DeepSeek Database Leaking Sensitive Information, Including Chat History (2025) — link
  42. Tax non-compliance detection using co-evolution of tax evasion risk and audit likelihood (2015)
  43. The general theory of tax avoidance (1985)
  44. Federal Taxation of Income, Estates and Gifts \textparagraph 4.3 (2025)
  45. Esmark, I nc. v. C ommissioner (1988)
  46. Notice 2003-54, COMMON TRUST FUND STRADDLE TAX SHELTER (2003)
  47. Common Trust Funds: The Living Fossil of Passthroughs (2010)
  48. Recognized abusive and listed transactions (2025) — link
  49. Large language models as tax attorneys: a case study in legal capabilities emergence (2024)
  50. Tax Gap Projections Methodology: Tax Year 2022 (2024)
  51. Base erosion and profit shifting (2025) — link
  52. Rulearena: A benchmark for rule-guided reasoning with llms in real-world scenarios (2024) — link
  53. The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas (2025) — link
  54. Shelter Check: Proactively Finding Tax Minimization Strategies via AI (2025) — link
  55. Intellectual Property Law Solutions to Tax Avoidance (2015)
  56. AI, Taxation, and Valuation (2023) — link

0 of 64 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.