AI Tax Research Reliability: What It Gets Right and Wrong

By Sana Qureshi  |  Reviewed by a CPA on the Finance Fundamentals editorial team for technical accuracy.

Coverage in this article focuses on U.S. federal tax research practice. State, local, and non-U.S. rules differ and are not addressed here.

A preparer at a mid-sized firm types one question into a general AI chatbot the night before a filing deadline. “What’s the current de minimis threshold for this election, and is there a ruling that supports it?” The chatbot answers fast, in full sentences, with a citation that looks exactly like the hundreds of real ones the preparer has read over a career.

It’s also wrong. Not wrong in an obvious way. Wrong in the specific, quiet way that survives a skim and fails a phone call to the clerk’s office.

That scenario isn’t hypothetical in the sense that it never happens. It’s a composite of a pattern documented again and again in court dockets, bar association warnings, and vendor research over the past three years. This guide covers what AI research tools do well, where they break down, and what a checking habit needs to look like in a tax practice.

Why AI Tax Research Spread Faster Than Rules Around It

Generative AI moved into tax and accounting workflows faster than most firms could write policy for it. Bloomberg Tax reports that 81% of Fortune 500 companies and 87% of the top 100 U.S. accounting firms already use its AI-integrated research tools in some form. Those figures come from a May 2026 vendor analysis of hallucination risk.

Read them as a floor, not an independent audit; vendor-reported adoption numbers usually run generous. The direction isn’t in dispute, though. AI-assisted research is now a normal part of how returns get prepared and positions get supported.

Adoption outpaced oversight. The IRS Office of Professional Responsibility didn’t issue its first dedicated guidance on the topic until June 24, 2026, in a document titled “Introductory Guidelines for Responsible AI Use in Federal Tax Practice.” It states plainly that existing duties of competence and diligence already cover AI-assisted work.

“Due diligence requires verifying the accuracy of facts, citations, and calculations produced by AI.” Blind reliance on unclear AI output, the guidance adds, is not a defensible professional judgment. Source: IRS Office of Professional Responsibility, June 24, 2026.

Legal research offers the clearest evidence of what happens without that check, because litigation creates a public record that tax practice mostly doesn’t. Stanford’s Institute for Human-Centered AI and its Regulation, Evaluation, and Governance Lab published a study in January 2024 testing general-purpose models, including GPT-3.5, Llama 2, and PaLM 2. The models hallucinated on legal questions between 69% and 88% of the time. On questions about whether a specific court ruling was still good law, the failure rate climbed above 75%.

A second Stanford paper, released as a preprint on May 28, 2024, tested purpose-built legal research products instead of raw models. Lexis+ AI hallucinated on roughly 17% of queries and answered accurately on about 65%. Thomson Reuters’s Ask Practical Law AI hallucinated at a broadly similar rate, but declined to answer 62% of the time, landing on a fully accurate response only 18% of the time.

Purpose-built tools clearly do better than raw chatbots. But 17% is still one wrong citation in every six queries. No tax practitioner would accept that failure rate from a junior associate without a second read. There’s no principled reason to accept it from software.

What AI Research Tools Actually Do Well

Retrieval and summarization are where these tools earn their keep. Ask one to pull the text of a specific Treasury regulation, a named revenue procedure, or a code section you can identify by number. Modern tools generally find it and summarize it in workable plain language. That’s a search-and-compress task, close to what a good full-text database with a language layer on top was built for.

They’re fast at first-pass organization too. Feed a tool a stack of source documents and ask for an issue outline, a comparison of two provisions, or a plain translation of dense statutory language. The output is usually a reasonable starting draft. It saves real time on the mechanical parts of research: finding, sorting, restating.

The common thread across both strengths is simple. These tools perform best when a human has already supplied the ground truth. Point one at a document and ask it to work inside that document, and it behaves like a decent research assistant. Ask it to go find the ground truth on its own, and the reliability picture changes fast.

Where AI Tax Research Breaks Down

Three failure modes account for most of the damage: fabricated authority, superseded law presented as current, and jurisdiction-specific nuance flattened into one generic answer.

Fabricated authority is the most visible failure, because it produces something checkable and false at the same time. A case name, a revenue ruling number, a private letter ruling, or a quote attributed to a real court that never said it. The citation looks correct in format, reporter, volume, page, and year all sitting where they should. The problem surfaces only when someone tries to pull the actual source and finds nothing there.

Superseded law is quieter, and arguably more dangerous in tax work, because it doesn’t announce itself as an error. Thomson Reuters’s tax and accounting group has pointed to the rollout of the One Big Beautiful Bill Act as a live example. IRS guidance on new OBBBA provisions was still being issued well into 2025 and 2026.

Any AI tool trained on older data answers with the same confident tone, whether the rule it names is current or a year stale. A wrong effective date doesn’t look like a hallucination. It looks like an answer, and a plausible one.

Jurisdiction-specific nuance is the failure mode general-purpose tools handle worst. Federal, state, and local tax rules genuinely diverge, and a model trained mostly on federal material tends to generalize across that divide anyway. A conformity date, a state-specific addback, or a local apportionment rule can flip an answer that was correct federally into one that’s wrong for the state return on the desk. The model rarely flags the uncertainty. It just answers.

Errors rarely look dramatic. As Thomson Reuters’s tax and accounting group has put it, a hallucination in tax research is more often “a threshold number that reflects a prior tax year” than an obviously invented case.

AI research features are also showing up bundled directly into adjacent workflows now, not just as a standalone search box. Provision software vendors are increasingly folding AI-assisted automated tax provision calculations and research suggestions into the same interface.

The idea is that a preparer working on ASC 740 positions benefits from research support inside the same screen. That convenience raises its own separate accuracy question. A hallucinated citation embedded inside a provision workpaper is arguably harder to catch than one sitting alone in a memo, dressed up as it is in the surrounding rigor of the calculation itself.

How AI Reliability Varies by Task Type

Reliability isn’t one number for a given tool. It depends heavily on what you’re asking it to do. The table below breaks down common tax research tasks by how much independent verification each one actually requires.

How Hallucination Rates Compare Across Tool Types

The most rigorous public numbers here come from legal research, not tax research, because no independent lab has yet published a matching benchmark for tax-specific AI products. That gap is worth noting on its own. Several vendors advertise accuracy claims for tax research features without pointing to independently reproduced testing. The chart below uses the legal research data as the closest available proxy for what “hallucination rate” looks like across tool categories.

Verification Practices Tax Professionals Should Build In

A workable checking habit doesn’t need to be elaborate. It needs to be consistent, applied the same way on a Tuesday rush job as on a marquee engagement.

• Start with the citation, not the conclusion. Before accepting an AI-generated answer, pull the actual ruling, code section, or case and read the relevant passage yourself. If the source can’t be located through the primary database your firm already subscribes to, treat that as a hallucination until proven otherwise.
• Check the date twice. Once for the source’s issue date, once for whether anything has superseded it since. AI tools are weak at signaling supersession, because a revoked ruling and an active one read identically in the training text.
• Separate drafting from sourcing. Let the tool help structure a memo or translate dense language. Then source every substantive claim on your own, the way you’d handle a draft from a first-year associate who hasn’t yet earned blind trust on citations.

A Documented AI Research Error, and How It Was Caught

The clearest illustration of this failure pattern in a filed proceeding is Gauthier v. Goodyear Tire & Rubber Co., decided in the U.S. District Court for the Eastern District of Texas and covered by the American Bar Association’s Litigation News in 2025. Plaintiff’s counsel filed a response to a summary judgment motion citing two cases that do not exist, along with quotations that couldn’t be located in any real authority.

The attorney later admitted he had filed the response “without reading the cited cases or even confirming their existence.” He had used one AI tool to draft the brief and a second AI tool to attempt to verify it. Both failed to catch the fabricated content.

Opposing counsel and the court caught what the attorney’s own process didn’t. The fabricated cases surfaced when the other side tried to pull them and came up empty. That was a five-minute check the filing attorney had skipped twice: once at drafting, once at the AI “verification” step that wasn’t actually verification. The court imposed a $2,000 sanction under Federal Rule of Civil Procedure 11(b)(2) and ordered one hour of continuing legal education on AI use.

The pattern repeats outside that single case. In Smith v. Farwell, decided by the Superior Court of Massachusetts on February 12, 2024, an attorney filed three pleadings with fictitious citations. He first blamed the errors on interns. He then admitted an AI tool had generated them, and that the pleadings were reviewed only “for style and grammar, but not for accuracy.” A $2,000 sanction followed.

Researcher Damien Charlotin’s public database of AI hallucination cases in litigation had logged 1,809 such incidents worldwide as of July 25, 2026, including one from an Indiana tax tribunal. This isn’t confined to civil litigation generally. It has already reached property tax disputes specifically.

Common Mistakes Firms Make With AI Research Tools

Over-trusting the output is the root mistake. It shows up in a few predictable shapes.

• Treating a fluent, well-formatted answer as evidence of accuracy. Formatting quality and factual accuracy are unrelated in a language model’s output, and a hallucinated citation is formatted exactly as cleanly as a real one.
• Using a second AI tool to “check” the first tool’s work, as happened in Gauthier. If both tools were trained on overlapping data and neither has access to a live, authoritative citation index, this checks nothing.
• Skipping the check under deadline pressure, exactly when the urge to skip it is highest and the cost of a wrong answer is highest too.
• Assuming a paid, purpose-built research product can’t hallucinate because it’s marketed as “trained on legal and tax content.” The Stanford data on Lexis+ AI and Ask Practical Law AI shows purpose-built tools still hallucinate at meaningful rates. Better than raw chatbots, not error-free.

Some AI-in-tax marketing describes these systems as capable of answering “any tax question with authority.” That claim overstates what any current tool can actually guarantee about a citation it hasn’t independently checked against a primary source. Firms that repeat it to clients are setting an expectation the underlying technology can’t currently meet.

A Practical Verification Checklist

Before an AI-assisted research answer goes into a memo, a return position, or a court filing, run it through a short, repeatable check.

• Pull the actual source cited and read the specific passage the AI relied on.
• Confirm the source’s current status: not superseded, revoked, or overruled.
• Confirm jurisdiction fit. A federal answer is not automatically a state answer.
• Confirm dates and thresholds against current-year figures, not the training data’s era.
• Document who verified the citation and when, inside the workpaper itself.
• Treat a second AI tool’s “confirmation” as no confirmation at all.

None of this erases the value of AI-assisted research. It puts the burden of checking where it has always belonged, with the licensed professional signing the work, not the tool that drafted it.

Key Takeaways

Frequently Asked Questions

Is AI tax research reliable enough to use without checking it?

No. Even purpose-built legal and tax research products with independent testing behind them have shown hallucination rates around 17% in published studies. General-purpose chatbots without a dedicated legal or tax database have tested far worse, up to 88% in one Stanford study. Every citation, ruling status, and calculated threshold needs an independent check before it’s relied on.

What is an AI hallucination in a tax research context?

It’s a confident, plausible-looking output that’s factually wrong: a case or ruling that doesn’t exist, a real ruling misquoted, or a rule described as current when it’s been superseded. The defining feature is that it reads exactly like a correct answer, which is why format-based skimming doesn’t catch it.

Have tax or legal professionals actually been sanctioned for this?

Yes, repeatedly. Courts sanctioned attorneys in Mata v. Avianca, Inc. in 2023 for ChatGPT-fabricated case citations, in Smith v. Farwell in Massachusetts in February 2024, and in Gauthier v. Goodyear Tire & Rubber Co. in 2025, among others. A public database maintained by researcher Damien Charlotin had tracked 1,809 such cases worldwide as of July 2026, including at least one property tax tribunal matter.

Do purpose-built tax and legal AI tools hallucinate less than general chatbots?

Meaningfully less, based on the available data, but not close to zero. Stanford’s May 2024 study found Lexis+ AI hallucinated on about 17% of queries versus 69%–88% for raw general-purpose models. No independent benchmark yet exists for tax research products alone, so vendor accuracy claims for tax tools should be treated as unproven until tested the same way.

Which tasks carry the highest AI research risk in tax practice?

Open-ended requests for citations, questions about whether a rule is still current on a recently changed provision, and multistate or local conformity questions carry the most risk. Tasks where you supply the source document and ask the tool to summarize or organize it carry the least.

What should a firm’s AI verification policy actually require?

At minimum, four things. Independent primary-source checking of every citation before it’s used in client work. A written record of who checked what, and when. A rule against treating a second AI tool’s output as a check. Clear sign-off resting with a licensed professional, not the software.

References

• Stanford RegLab / Stanford HAI, “Hallucinating Law: Legal Mistakes with Large Language Models are Pervasive,” January 2024.
• Stanford RegLab, “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools,” preprint, May 28, 2024.
• Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023).
• Smith v. Farwell, Superior Court of Massachusetts, Civil Action No. 2282CV01197, opinion Feb. 12, 2024.
• American Bar Association, Litigation News, “Lawyer Sanctioned for Failure to Catch AI ‘Hallucination'” (Gauthier v. Goodyear Tire & Rubber Co.), 2025.
• Damien Charlotin, AI Hallucination Cases Database, accessed July 25, 2026.
• Internal Revenue Service, Office of Professional Responsibility, “Introductory Guidelines for Responsible AI Use in Federal Tax Practice,” June 24, 2026.
• Bloomberg Tax, “AI Hallucinations in Tax: The Risks and How to Mitigate Them,” May 11, 2026 (citing a January 2026 study in The International Tax Journal documenting roughly 800 AI-related citation errors across at least 25 countries).
• Thomson Reuters Tax & Accounting, “Why General AI Tools Fail at Professional Tax Research.”
• AICPA & CIMA, Artificial Intelligence (AI) Tax Resource Center.