AI contract chatbots for unions: what's real, what's risk

Somebody on your executive has asked about it, or will by the next meeting: should the local put a chatbot on the collective agreement so members can ask it questions? The pitch is obvious. The objections are obvious too, and most of them are right. What's missing is a straight account of which risks are real, which are solved by design, and which are only solved by asking the vendor the questions they'd rather you didn't.

This is that account. It's written for the president, business agent, or counsel who has to sign off, and it's built to be forwarded to whoever is going to say no. Where we cite a number, the source is linked. Where we describe how a tool should work, we say what that means for our own product, so you can hold us to it.

What a collective agreement chatbot actually is — and the three versions of it

A collective agreement chatbot is a question-and-answer tool that reads a union's collective bargaining agreement and returns the relevant language when a member or steward asks about it in plain words. The useful ones cite the article and section. The dangerous ones don't.

The label hides three very different things, and most of the risk conversation goes wrong because it treats them as one:

The first column is what most members are already using. In Gallup's Q4 2025 workforce survey of 22,368 U.S. employees, 26% said they use AI at work a few times a week or more and 12% use it daily (Gallup, January 2026). Some of those questions are about the contract. The realistic choice for a local isn't "chatbot or no chatbot." It's whether the contract questions members are already typing go into a tool the union controls or one nobody does.

The second column is where the labour movement's alarm belongs, and we'll come back to it. The third is the one your executive is being asked to evaluate. The rest of this post is about the risks that apply to it, in the order counsel usually raises them.

Risk 1: Hallucination — the tool says something the agreement doesn't

This is the risk everyone names first, and it deserves to be. Large language models produce fluent text whether or not it's true. The U.S. National Institute of Standards and Technology calls the failure confabulation: a system that will "generate and confidently present erroneous or false content" (NIST AI 600-1, July 2024). In a contract tool, that's a member being told they're owed a premium the agreement doesn't grant — or, worse, told they aren't when they are.

The evidence on how often this happens is better than most vendors admit. Stanford researchers asked general-purpose GPT-4 questions about real, verifiable U.S. federal cases and found it hallucinated 58% of the time; GPT-3.5 hallucinated 69% and Llama 2 88% (Dahl et al., Journal of Legal Analysis, 2024). A follow-up study by the same group tested purpose-built legal research tools that ground answers in a document set — the technique that matters here — against more than 200 hand-labelled legal queries. Grounded tools were markedly better than the open-ended chatbot but still not clean: the tools from LexisNexis and Thomson Reuters "each hallucinate between 17% and 33% of the time" (Magesh et al., Journal of Empirical Legal Studies, 2025).

View as table

Tool	Accurate	Incomplete / declined	Hallucinated
GPT-4 (open-ended)	49%	8%	43%
Westlaw AI-Assisted Research	42%	25%	33%
Lexis+ AI	65%	18%	17%
Ask Practical Law AI	20%	63%	17%

Two things in that chart matter for a union. First, grounding the model in a fixed document set cut the hallucination rate by a quarter to more than half against the same questions — that's the argument for restricting a tool to your agreement and nothing else. Second, look at the last row: the tool with the lowest hallucination rate got there mostly by declining — 63% of its answers were incomplete or refused. That is not a flaw. For a contract tool, it's the design goal. A tool that says "that isn't in the agreement — ask your steward" is safe. A tool that fills the gap is not.

None of this is abstract in 2026. In Mata v. Avianca, a federal judge sanctioned two lawyers and their firm $5,000 for filing a brief with six cases ChatGPT had invented (S.D.N.Y., June 2023). That was the first widely reported case. A database maintained by legal researcher Damien Charlotin counted 10 court rulings involving AI-fabricated material in 2023, 37 in 2024, and 73 in the first five months of 2025 alone (Charlotin database, as reported by Business Insider, May 2025); by June 2026 the New York State Bar Association reported the same database had passed 1,000 U.S. cases (NYSBA, June 2026). Those are lawyers, with training and time to check. Members at 11 p.m. have neither.

View as table

Period	Rulings
2023	10
2024	37
January–May 2025	73

What actually mitigates it. Three things, all checkable in a demo:

1. Closed source of truth. The tool answers only from the agreement (and whatever letters of understanding, side letters, and MOUs the union loads). Not policy manuals it found online, not provincial employment standards, not a similar union's contract.
2. A citation on every answer. Article and section, every time, so a steward can verify in the time it takes to open the PDF. If the answer can't be traced to a clause, it shouldn't be given.
3. Refusal as a feature. When the question isn't in the agreement, the tool says so and points to the steward. Ask to see it decline something in the demo. If it can't, that's your answer.

For example (a composite drawn from contract work we see regularly): a member asks whether working a statutory holiday earns time-and-a-half plus a lieu day, or just time-and-a-half. A general chatbot answers confidently — from the province's employment standards, which set a floor the agreement exceeds. It's a plausible answer and it's wrong for this member. A grounded tool finds article 21.04, quotes the actual entitlement, cites it, and — if the agreement is silent on how lieu days are scheduled — says that part is one for the steward. The difference isn't intelligence. It's what the tool is allowed to read and what it's required to show.

For the fuller version of why a general-purpose chatbot is the wrong instrument for a contract, we've laid it out side by side on why not just use ChatGPT.

Risk 2: Employer visibility — could management see what members ask?

This is the risk that should decide the question, and it is almost entirely a matter of who deployed the tool.

The documented harm pattern in 2026 is employer-side AI. In an April 2026 survey for the AFL-CIO, 94% of workers said they should be told when AI is used to monitor their work — but 70% said their employer has never disclosed whether it does, and only 7% said it has (David Binder Research for the AFL-CIO, reported by The Stand, May 2026). Model contract language collected by the UC Berkeley Labor Center exists precisely because of this: one nurses' clause requires that, where a duty to bargain is triggered, the union "will be notified in advance of any new or changed monitoring or surveillance programs," with a definition covering keystroke monitoring, email access monitoring, and browser history retrieval (Penn State School of Labor and Employment Relations, 2025).

So the fear that a member's question — "can they discipline me for refusing the overtime?" — could end up in a manager's dashboard is well founded. For an employer-deployed assistant, it isn't a risk; it's the product. For a general chatbot on a work device, it depends on the employer's monitoring, which 70% of workers say they've never been told about.

For a union-deployed tool, the test is architectural, and it has to be pass/fail: the employer is not a party. Not "the employer has agreed not to look." No account, no login, no data feed, no relationship with the vendor. If the employer sits anywhere in the diagram — as a payer, a co-signer, an integration, a "read-only" viewer — the arrangement is not union-controlled and members should be told so.

That's the standard we hold ourselves to. On our security and privacy page, the employer question is item one, and the answer is that the employer has no access, no visibility, and no relationship with AviChat. It isn't a policy that could be renegotiated. There is no path for it. Ask any vendor for the equivalent sentence, in writing, before you go further.

Risk 3: Data sovereignty — where do the questions and the agreement actually live?

Every question a member types is a record. It says who was worried about what, when. Multiply that by a bargaining unit and it's a map of the local's pressure points — the sort of thing that would be very interesting to the other side of the table, and to anyone who ends up with a subpoena.

Two facts about consumer tools make counsel sit up. First, a consumer chatbot keeps conversations under the vendor's terms, not the union's — and those terms are the vendor's to change. Second, retention isn't fully within the vendor's control either. In May 2025 a federal magistrate judge in the New York Times v. OpenAI copyright case ordered OpenAI to "preserve and segregate all output log data that would otherwise be deleted on a going forward basis" — including logs from ChatGPT Free, Plus, and Pro users, whether or not those users had deleted their chats (In re OpenAI Copyright Infringement Litigation, No. 25-md-3143, ECF 559, S.D.N.Y., May 16, 2025). A member's "deleted" question about a pending discipline was, for a period, preserved by court order in another country. That is what "consumer terms" means in practice.

Canadian law puts the accountability squarely on the organization that collected the data — in this case, the union. The Office of the Privacy Commissioner's guidance is blunt: "an organization is responsible for personal information in its possession or custody, including information that has been transferred to a third party for processing," and it must be transparent that data sent to another jurisdiction "may be accessed by the courts, law enforcement and national security authorities" there — "no contract can override" that (OPC, Guidelines for processing personal data across borders). Quebec goes further: since September 2023, section 17 of Law 25 requires a privacy impact assessment and a written contract before personal information leaves the province (BLG summary of Law 25 s. 17, updated 2024). Public-sector locals often face additional public-body privacy rules on top of this — check with counsel.

A tool built for unions should be able to answer all five in one sentence each. If a vendor needs a call to answer any of them, that's a data point too.

Risk 4: Member privacy — can the union see who asked what?

This is the risk unions ask about least and should ask about most, because it's the one that turns on the union's own conduct.

Leadership does want to know what members are asking; that's legitimate and it's useful. If a third of the questions in March were about the new attendance-management policy, that's bargaining intelligence you can't get from a survey. Topic-level analytics are counts and trends — how many questions, on which articles, moving which way. Individual lookup is pulling up what a named member asked on Tuesday. The line runs between the two.

The second must be impossible, not just prohibited. Not "we don't run that report." No such function exists. The reason is the same one that applies to the employer: a policy can be quietly amended after the next election, by someone with a grudge or a good excuse. Architecture can't. Members who suspect their steward can read their questions will stop asking, and the tool becomes a very expensive PDF viewer.

CUPE's own guidance to members makes the general point about workplace technology: "left unchecked," it warns, AI and other digital tools "can enable surveillance" — and workers' data is what powers it (CUPE, Understanding Artificial Intelligence: A guide for CUPE members). A union deploying its own tool doesn't get an exemption from that logic. It gets an obligation to be the counter-example.

Risk 5: Authority — who decides what a clause means?

There's a fifth concern that isn't usually named as a risk but sits under all the others: what happens when the agreement is genuinely ambiguous, and who the tool defers to.

Every agreement has provisions the parties read differently. Some have been arbitrated. Some are held in a truce nobody wants to test. A tool that picks a side on its own — even the union's side — is making an interpretive claim the union hasn't authorized, and a member could act on it. The right behaviour is procedural: ambiguous provisions get identified before launch and defined with the union's officers, in the union's words. What the tool then says is what the union has decided it should say. If nothing has been decided, the tool says so and sends the member to the steward.

This is also where the "is it replacing us?" question resolves. It doesn't. The tool does the lookup — what article 21.04 says — and hands the judgment work back to people. We wrote about that at length in Will AI replace union reps? What stewards should actually watch for, and the checklist there is a good companion to this one.

What's real: the upside, honestly bounded

After five sections of risk, it's fair to say what the tool is actually for, without a sales voice.

The case is narrow and it's real. Members ask the same few dozen questions, at every hour, and they land on the two or three people who always answer. Most of those questions are lookups: which article, what rate, what deadline. A well-built tool answers those in seconds with a citation, and the steward's phone rings for the things that need a steward. That's the whole value proposition. It doesn't organize, it doesn't file, it doesn't argue at step two.

The labour movement is not, in fact, refusing the technology; it's setting terms for it. NewsGuild-CWA has ratified 58 contracts with language governing AI in newsrooms, and CWA's agreements at Microsoft-owned ZeniMax "require the company to provide notice to the union when the implementation of AI technology impacts work performed by bargaining-unit employees" (CWA, March 2026). Workers, for their part, trust unions more than employers or either political party to set the rules: unions were the only institution with net-positive trust on AI in the AFL-CIO's April 2026 polling, at +26 (The Stand, May 2026). A union that deploys its own tool, on its own terms, is exercising exactly that trust — and demonstrating what the terms should look like when it's the employer's turn.

What a union should demand from any vendor: the eight-point checklist

Take this to the demo. Every item is a yes/no, and the vendor should be able to show it, not describe it. We've written it to be tool-agnostic; the note under it says how we answer.

For transparency: this is the bar we built AviChat to meet — closed to the agreement, cited on every answer, employer locked out by architecture, data in-country in an isolated instance owned by the union, never used to train anything, analytics anonymized and topic-level only, an audit trail on every answer backed by liability insurance, and ambiguous provisions defined with officers before launch. (The audit trail records what the tool said and which clause it cited — not who asked — and it is restricted to legal verification.) Counsel can check each claim in the two-page security overview. But run the list on anyone. If a vendor passes all eight, the risk conversation is largely over. If they pass six, you know exactly what to negotiate.

If you'd like the risk questions above answered against your own agreement, a walkthrough is fifteen minutes and there's no form to fill in first. If your CBA isn't a fit, we'll say so.