The so-called black box problem describes the difficulty of understanding how a complex system reached a conclusion, even when all the inputs are known. More than a decade into the age of modern artificial intelligence and several years into the public availability of generative AI, that problem is more prevalent and more consequential – particularly when government agencies use automated decision systems backed by increasingly sophisticated AI models. The concern is not technological novelty for its own sake. It is that a black box undermines transparency, accountability, and trust, which are critical to public acceptance of a core governmental function. In tax administration, that public acceptance is reflected in voluntary compliance, and voluntary compliance is foundational to U.S. federal and state taxation.

The image of a seasoned auditor selecting a taxpayer for audit based on industry experience or an intuition for spotting large-dollar adjustments now seems quaint. Federal and state tax authorities have relied on statistical analysis and computerized algorithms for decades. The IRS began using the Discriminant Function System (a statistical scoring model that compares a return against expected norms from other returns) to help select returns for audit in the 1960s. States likewise have used data analytics and, more recently, machine-learning tools for audit targeting. The broader pattern is clear: audit selection has moved from rule-based criteria and human judgment toward predictive analytics and machine-assisted targeting at both the federal and state levels.

The move to generative AI audit selection appears to be underway. The environment is materially different now, however: tax agencies have years of return information, enormous data sets (like transaction-level sales data), third-party information reports, non-tax agency databases, an ocean of publicly available information through the internet, multi-state information sharing arrangements, and, in general, fewer actual human beings available to oversee the process.

The challenge is not tax administration use of algorithms. Indeed, these systems have been instrumental in combatting refund fraud and taxpayers benefit if analytics reduce unnecessary audits of those who are compliant. The challenge is the absence of transparency, the obfuscation of accountability, and the erosion of trust. The challenge is the black box. No taxpayer should receive an audit notice that effectively says the taxpayer was selected by an unexplained automated system that no remaining human at the agency can meaningfully defend.