With the increasing popularity of artificial intelligence (AI), many people use AI tools like ChatGPT for guidance. Some even use it for sensitive smatters, including taxation, without fully understanding the attendant risks related to security, accuracy, and legal protection.
Are you using AI as a tax advisor or consultant? If so, you should reconsider sharing sensitive information on online platforms.
No Legal Protection for Information Shared With AI
In a recent interview with podcaster Theo Von, Sam Altman, the CEO of OpenAI, acknowledged the risk of sharing sensitive information with OpenAI’s ChatGPT,1 currently the most used AI tool. Altman recognized the lack of legal protection for sensitive information shared with AI tools, unlike information shared with human professionals like therapists, lawyers, and doctors, who are legally bound by client-confidentiality clauses. He further stated that if a privacy matter ends up in legal proceedings, AI providers could be compelled to hand sensitive information to the authorities—a possibility Altman clearly does not look forward to:
People talk about the most personal details in their lives to ChatGPT. People use it, young people especially, use it as a therapist, a life coach, having these relationship problems and [asking], What should I do? And right now, if you talk to a therapist or a lawyer or a doctor about those problems, there’s legal privilege for it. There’s doctor-patient confidentiality, there’s legal confidentiality, whatever. And we haven’t figured that out yet for when you talk to ChatGPT. So, if you go talk to ChatGPT about your most sensitive stuff and then there’s a lawsuit or whatever, we could be required to produce that. And I think that’s very screwed up. I think we should have the same concept of privacy for your conversations with AI that we do with a therapist or whatever. And no one had to think about that even a year ago, and now I think it’s this huge issue of, like, how are we going to treat the laws around this?
Altman’s statement clearly highlights the risk of sharing sensitive information with AI, given the possibility that the information may one day end up being aired in court. This risk applies to all sensitive matters, including taxation, that one might disclose to AI. Especially in complex tax matters, to receive meaningful advice users often must share highly detailed personal and sensitive information. AI can provide rule-based advice, often for free; however, it is very risky to share detailed information due to the lack of legal protection—information that can also be used against you.
AI Can Be Unreliable and Misleading
In another instance, Altman, speaking at a banking conference hosted by the US Federal Reserve, warned against relying too much on ChatGPT for decision-making.2 He stated that many people believe that AI can give better advice than humans can due to its ability to process large volumes of information quickly. However, relying heavily on AI can be dangerous.
A notable example is the case of Mata v. Avianca, Inc., where a New York–based law firm used ChatGPT to prepare a document for a client’s personal injury claim against an airline.3 In its filing, the law firm cited many past cases to support its argument. However, the court found that six of the cited cases did not exist. The case quotations were also bogus. The law firm admitted to the court that it had used ChatGPT to prepare the case file. The firm not only lost its credibility but also faced the heat of sanctions from the court.
Overreliance on AI can also impair human intellectual capability. Individuals who depend on AI may struggle to interpret information without it and may need AI assistance even for routine work. In the US Tax Court case Dealers Auto Auction of Southwest LLC v. Commissioner, an Arizona company purchased specialized software to assist in preparing Internal Revenue Service Form 8300, Report of Cash Payments Over $10,000 Received in a Trade or Business.4 The company relied on tax software to file all required forms for its Form 8300 return. Despite the fact that Dealers Auto had the software in place, the IRS determined that the company had submitted only 116 of 382 required Forms 8300 for tax year 2016, resulting in $118,140 in penalties. The court ruled that reliance on software is not a reasonable defense when the taxpayer fails to input the correct information. Although this case emerged several years before the advent of ChatGPT and other AI tools, it nonetheless underlined the important role of human error and judgment, even in the use of advanced technological tools.
AI Struggles With Complex Tax Scenarios
In 2024, the Washington Post investigated the AI capabilities of two leading tax software tools, TurboTax’s Intuit Assist and H&R Block’s AI Tax Assist.5 The reporter, Geoffrey A. Fowler, posed questions regarding filing recommendations for when a child attends college out of state and for cryptocurrency reporting. The findings revealed that both AI-powered chatbots gave misleading and inaccurate information, even in only moderately complex tax situations, leading Fowler to caution taxpayers against using AI tools.
Similarly, the IRS’ Taxpayer Advocate Service cautions taxpayers against dependence on AI for tax matters. In an official statement, the service stated, “Despite efforts to ensure accuracy, these AI assistants may encounter difficulties interpreting complex tax laws correctly or considering unique circumstances that could impact a taxpayer’s return. As a result, taxpayers should not solely rely on AI-generated tax advice.”6
How Should AI Be Used in Tax?
AI is now central to all major tax technology innovations. It is impossible to avoid using it. However, we can take smart precautions to protect sensitive data and take full advantage of technological advancements.
Avoid Sharing Sensitive Information With General AI Tools
First and foremost, avoid sharing sensitive information with general-purpose AI tools like ChatGPT. These open AI tools are generally free, but they are not governed by any law to protect the shared information. Indeed, most AI tools do not reveal the security measures they use to protect the data shared with them. What’s more, most tools use shared data to train AI models. As a result, any sensitive information shared with these tools can potentially be accessible to others, and even used against you.
Use Secure, Specialized AI Tax Tools—With Caution
Specific AI-based tax tools offer better protection for sensitive information, but the security depends on several factors that users should consider, like:
- Encryption. What encryption technology does the tax tool use to protect data in transit (TLS/SSL) and at rest (AES-256)?
- Access controls. Who can access the system? Is access role-based or open? Does the system support multifactor authentication?
- Security and audit certificates. Does the provider hold independent security and audit certificates like SOC 2 or ISO 27001, or does it comply with IRS Publication 1075, Tax Information Security Guidelines?
- Storage facilities. Where is the data stored (for example, Amazon’s AWS or Microsoft Azure)? What level of protection is available against cyberattack?
Most important, ensure that your data is not being used to train AI models. Many data leaks stem from this practice. Make sure that the provider explicitly certifies that no customer data is used for AI model training.
Follow Best Practices
Precautions are always wise. Consider these best practices when working with AI-driven software.
- Limit sharing sensitive identifiers (SSNs, passport numbers, banking information). Until and unless it is absolutely necessary, avoid sharing sensitive information. If needed, share it only for stated purposes, like filing a secured return;
- Use AI for guidance, not for final filing decisions. Always verify results with a certified tax professional like a certified public accountant, chartered accountant, or tax attorney or directly against official tax regulations. Don’t rely on AI alone; and
- Monitor legal privilege developments. Currently, no law protects information shared with AI. However, AI is still relatively new, and governments are only now drafting or amending laws for its safe use. Monitor and stay informed about government AI regulations.
Conclusion
AI can be helpful for general guidance but not as a substitute for professional tax advice. The risks to security, accuracy, and legal protection are real and significant. AI tax tools can provide strong technical security (encryption, cloud protection, compliance) but do not guarantee legal confidentiality or immunity from breaches or misuse. Until laws catch up, the safest practice is to treat AI tax chats as unprivileged conversations, useful for general guidance but not for unfiltered disclosure of sensitive financial details. Use AI wisely, and always consult a qualified expert for complex or sensitive matters.
Avnish Goyal is a senior IT solution architect at Ortho Clinical Diagnostics.
Endnotes
- Sam Altman, “Interview With Sam Altman,” by Theo Von, July 23, 2025, www.youtube.com/watch?v=aYn8VKW6vXA.
- “Sam Altman Speaks at Federal Reserve Conference,” Electron Media Group, July 22, 2025, www.youtube.com/watch?v=GhyCo-Ug5tY.
- Kathryn Armstrong, “ChatGPT: US Lawyer Admits Using AI for Case Research,” BBC News, May 27, 2023, www.bbc.com/news/world-us-canada-65735769.
- “Reliance on Software Doesn’t Excuse Reporting Failures,” Tax Notes, April 28, 2025, www.taxnotes.com/research/federal/court-documents/court-opinions-and-orders/reliance-software-doesnt-excuse-reporting-failures/7s44l.
- Geoffrey A. Fowler, “TurboTax and H&R Block Now Use AI for Tax Advice. It’s Awful,” Washington Post, March 4, 2024, www.washingtonpost.com/technology/2024/03/04/ai-taxes-turbotax-hrblock-chatbot/.
- “Is AI-Generated Tax Advice Making the Grade?” Internal Revenue Service, Taxpayer Advocate Service, June 11, 2024, www.taxpayeradvocate.irs.gov/news/tax-tips/is-ai-generated-tax-advice-making-the-grade/2024/06/.
