A public database maintained by legal researcher Damien Charlotin has been tracking every instance a court has caught a party citing AI-fabricated material in a filing. By mid-2026 the count had passed a thousand U.S. decisions alone, and the pace is accelerating as more lawyers adopt the tools. The penalties are no longer symbolic. Courts have issued fines in the tens of thousands of dollars against individual firms, the Sixth Circuit imposed a $30,000 sanction on two attorneys under the stiffest penalty its own rules allow, and at least one lawyer has lost the ability to practice in a district after fabricated citations turned up in a filing. The American Bar Association's Formal Opinion 512 makes the underlying point explicit: a lawyer's duties of competence and candor to the court do not soften because AI produced the draft. The person who filed it answers for what's wrong with it.
The mechanism behind these failures isn't specific to legal writing. A language model asked to produce a citation, a statistic, or a supporting quote will, when it doesn't have one, generate something that reads exactly like a real one — correct format, plausible name, a number that looks precisely calculated rather than invented. Nothing about the output signals uncertainty. That's what made this a courtroom crisis rather than a minor annoyance: a fabricated citation and a real one are indistinguishable to a reader who doesn't independently check it, and a lawyer under deadline pressure, trusting a tool that had been right the last dozen times, frequently didn't.
That same mechanism operates in every business document a language model helps draft, and law is simply where it surfaced first and most visibly, because litigation forces a filing into an adversarial process with a judge positioned to check it against the record. A due diligence memo citing a regulatory requirement that doesn't exist in the form described. A grant application referencing a study that was never published. An insurance claim narrative citing a policy provision the model paraphrased into something the policy doesn't actually say. A market-sizing slide in an investor deck built on an industry statistic that sounded like the kind of number that belongs in that sentence. None of these get the scrutiny a judge applies to a legal brief. Most get read once, by someone who trusts the person who wrote them, and then get acted on.
The exposure this creates for an ordinary business is the one courts have been enforcing against law firms: the business that submits the document owns what's wrong with it, regardless of which tool produced the draft. A lender, a regulator, a grant committee, or an insurer evaluating a submission has no reason to distinguish between a business that lied and one that let an AI tool invent something on its behalf and never checked. The consequence — a rejected claim, a voided grant, a due-diligence finding that sinks a deal — lands on the business either way. A related version of this has shown up in customer-facing chatbots, where courts have held that what a company's AI tool tells a customer counts as the company's own statement. The same logic reaches internal work product the moment it leaves the building under the business's name.
The fix courts have effectively imposed on the legal profession — verify every citation against the actual source before filing it — generalizes cleanly to any business document with real stakes attached. Not a ban on using AI to draft; a rule that a fact, figure, quote, or reference an AI tool produced is unverified until someone checks it against a primary source, the way a competent professional would check a junior colleague's first draft before putting their own name on it. The habit that fails is treating fluent, confidently-worded AI output as though its confidence were evidence of accuracy. It isn't. The tool sounds exactly as sure when it's fabricating as when it's right.
The practical version of this is narrower than a full compliance program. Identify which documents in your business carry real consequences if they contain a wrong fact — regulatory filings, insurance claims, loan applications, grant submissions, board materials, anything with a citation or statistic a reader might rely on or an auditor might later check. For those specific documents, build in one deliberate step before they go out: someone traces every factual claim, number, and cited source back to where it actually came from, not back to the AI tool that phrased it. That step takes minutes on a short document and is exactly the step every sanctioned filing in the tracker skipped.
None of this argues for pulling back from using AI to draft business documents, which remains a genuine time saver for the kind of writing most businesses do constantly. It argues for treating the draft as a draft — fast, useful, and unverified — rather than a finished deliverable simply because it read cleanly on the first pass. The businesses that avoid becoming the next entry in a sanctions tracker, or its quieter commercial equivalent, will not be the ones that stopped using AI to write. They will be the ones that never let a sentence with a fact in it leave the building without someone checking that the fact was real.
- ai hallucination
- due diligence
- professional liability
- document verification
- ai governance