When an AI agent makes a costly mistake, most enterprise contracts leave the customer holding the risk. Vendors are building agents that act on their own, while their terms make customers responsible for checking outputs and limit what vendors owe when things go wrong. Before an agent touches financial systems or customer records, settle accountability, liability caps, oversight, audit trails and remedies.
Two pieces of AI news landed within two days of each other this week, and they point in opposite directions.
On October 21, Anthropic introduced Claude Duet, an assistant that works across your connected apps, watches for what needs doing and keeps working after you step away. Two days earlier, CIO reported that Meta’s revised commercial terms, taking effect October 30, put the risk of its AI features and whatever they produce on the customer.
The two aren’t connected. Read together, though, they show where enterprise AI is heading. The products are being built to act on their own, and the contracts are being written so the vendor isn’t on the hook when they get it wrong.
I’ve been on the buying side of this. At YMCA Canada I ran a $15M technology portfolio across 37 associations, including one of the first enterprise AI pilots in the federation with Microsoft Copilot and ChatGPT. The question I’d be asking today is less about what these agents can do and more about who pays when one of them makes a mistake.
What Duet actually does
Duet runs on Claude Opus 5.5 and has its own cloud-based computer. It can prepare briefings, build dashboards and carry out multi-step tasks without someone walking it through each one. Anthropic says it will ask for approval before anything consequential, like sending a message or making a payment, and it has added safeguards around credentials and sensitive data.
Those are real protections and I’d want them. But an approval step is only as good as what the person approving can see. By the time Duet asks, it has already gathered the information, interpreted it and lined up the decision. Most people will click yes.
Duet is a research preview for US Pro and Max subscribers, so nobody is putting it into accounts payable next week. It does show what every vendor is building toward.
Meanwhile, in the fine print
Meta isn’t an outlier. OpenAI’s business terms make the customer responsible for checking whether outputs are accurate and appropriate before using them. Google Cloud’s service terms warn that generative AI can produce inaccurate content and isn’t meant to satisfy your legal or regulatory obligations. Microsoft’s AI Services Code of Conduct expects customers to put controls around AI outputs, decisions and autonomous actions. Anthropic’s own past commercial agreements have disclaimed any warranty that outputs will be accurate or error-free.
Fair enough, these aren’t identical. Some assign duties to the customer and others disclaim warranties or cap liability. Several vendors do offer indemnification, mostly for intellectual property claims. The pattern is still hard to miss. The products are taking on more of the work while the terms leave the consequences with you.
The $100,000 invoice
Picture an agent running accounts payable. It reviews invoices, matches them to purchase orders and prepares payments. One day it misreads an invoice and recommends sending $100,000 to the wrong recipient, and a manager approves it based on the agent’s analysis.
Who is responsible? The manager who clicked approve? The organization that deployed the agent? The company that built the workflow? The provider whose model misread the invoice?
Swap in an agent that leaks confidential customer data, or makes a bad call about an employee, and the questions get harder. Traditional enterprise software contracts have settled ways of handling warranties, service levels and remedies. AI breaks some of that because the output isn’t predictable and the system is now doing work a person used to do. A disclaimer doesn’t erase every legal obligation a vendor has, but it can make it much harder for you to recover what you lost.
What I’d want settled before an agent touches anything important
AI governance usually means cybersecurity, privacy and acceptable-use policies. Contract risk needs to be on that list too, and it needs to be dealt with before an agent gets access to financial systems or customer records:
1/ Accountability. If the model fails, the integration breaks or the agent does something it wasn’t authorized to do, which party owns it?
2/ Liability. Are the contract’s liability caps anywhere near the size of the financial and operational risk you’re taking on?
3/ Oversight. Which decisions need a human, and what does that person need to see for the approval to mean anything?
4/ Audit trail. Can you reconstruct what the agent looked at, what it decided and what it did?
5/ Remedies. What happens when the system doesn’t perform the way the vendor said it would?
There is nothing unreasonable about expecting customers to use AI responsibly. You should be watching the tools you deploy. But vendors are now selling these systems as something close to a digital employee, and I think they’ll have to carry more of the risk if they want enterprises to hand over consequential work. The procurement teams that start asking for it now will get better terms than the ones who wait for the first big loss to make the case for them.
Frequently Asked Questions
What is Claude Duet?
Claude Duet is an assistant Anthropic introduced on October 21, 2026. It runs on Claude Opus 5.5 with its own cloud-based computer, works across your connected apps and keeps working after you step away. It asks for approval before consequential actions like sending a message or making a payment. For now it’s a research preview for US Pro and Max subscribers.
What did Meta change in its enterprise AI terms?
According to CIO, Meta’s revised commercial terms, taking effect October 30, put the risk of its AI features and whatever they produce on the customer. Meta isn’t alone. OpenAI, Google Cloud and Microsoft terms also leave much of the responsibility for checking and controlling AI outputs with the customer.
Who is responsible when an AI agent makes a mistake?
Under most current vendor terms, mostly the customer. A disclaimer doesn’t erase every legal obligation a vendor has, but it can make it much harder to recover what you lost. That’s why accountability needs to be settled in the contract before an agent gets access to anything important.
What should procurement teams ask before deploying an AI agent?
Who owns a failure, whether the liability caps match the risk you’re taking on, which decisions need a human and what that person gets to see, whether you can reconstruct what the agent did, and what remedies apply when the system doesn’t perform as promised. Teams that ask now will get better terms than the ones who wait.
