One of the ClioCon sessions I flagged ahead of time was Privilege, Supervision & AI in the Modern Law Firm. It turned out to be one of the most practical hours of the conference.
The speakers came at it from two sides. Georgia Horstman is a compliance analyst at Clio who works on AI. John Hendricks is a cyber law litigator in San Francisco who’s licensed in California and as a solicitor in England.
AI is a very capable first-year associate
John opened with a framing he credited to Clio’s Ed Walters. Treat generative AI like the world’s most capable first-year associate. Not the best one, the most capable.
It’s fast and never gets tired. But it sometimes makes things up, sometimes misreads your instructions, and it has no judgment. So your job is the same as always: train it, instruct it, and supervise it. Georgia added that this associate never becomes a second-year. You supervise it forever.
A court already tested this
In February, a federal court in New York decided United States v. Heppner. A CEO facing fraud charges used the consumer version of Claude to analyze his case, on his own and without his lawyers directing it. The court held those documents weren’t privileged and weren’t work product.
Part of the reasoning: the AI provider’s privacy policy undercut any expectation that the conversations were confidential. It’s one district court ruling, and it didn’t address enterprise tools or AI use directed by a lawyer. But it shows that the tool’s terms matter.
Five questions to ask any AI vendor
Georgia asks every AI provider these:
Do you train on my data?
How is my data kept separate from other users, and can you honor my ethical walls?
Do your staff read my files or outputs? For many lawyers, that’s a deal-breaker.
Where is my data processed, not just stored? Does it leave my jurisdiction?
What happens to my data when I leave? Can you give me a certificate that it’s destroyed?
Where to find the answers
The vendor’s trust website. Many vendors have one at trust.vendorname.com. Clio’s is trust.clio.com.
The SOC 2 Type 2 report. It’s an outside audit of the vendor’s security controls. Ask what’s changed since it was issued, and read the section near the end that lists what the vendor expects you to do.
The subprocessor list. It shows which other companies get your data and where they keep it.
The contract. Georgia’s sharpest point was that a help center article isn’t a contract. Look for zero data retention in writing, for the vendor and its subprocessors. And keep a dated record of what the vendor told you.
In the loop, or on the loop?
The second half covered what comes out of AI.
Human in the loop means the AI stops and asks for your approval at each key step. John’s example was drafting a reply brief, going back and forth to find and test authority.
Human on the loop means the AI acts on its own. Think of an AI that talks with a prospective client and books the consult. That’s fine for low-risk work, as long as the tool keeps a log you can review.
Either way, the responsibility is yours. A contract can shift some financial risk to a vendor. It can’t shift your professional duties.
What I’d take from it
Write down which AI tools your firm uses, and for what. John keeps a register of the tools he’s vetted. It also catches the tools people use without telling anyone.
Ask Georgia’s five questions before you sign up. Then check the answers against the contract.
Make checking AI’s work easy. Georgia’s view is that if verifying is a hassle, people skip it. Ask for inline citations you can click, and tell the AI to say when it isn’t sure.
The room was split when Georgia asked whether people could confidently tell a client where their data goes when it’s put into an AI tool. That’s a good question to be able to answer before a client asks it.
;-)
Ernie
P.S. In the Inner Circle, members compare notes on which AI tools they’ve vetted and what they found, so you don’t have to read every vendor’s fine print alone.
→
https://innercircle.ernietheattorney.net/