Our Civic Assistant has now had ten million conversations. In most of them, it routes, explains, or drafts — all low-stakes. But in a growing share, it sits between a resident and an outcome that matters. This post is about the rules that govern those moments.
The problem with confident AI in public service
An AI that sometimes gives wrong answers in a chat app is a productivity issue. An AI that sometimes gives wrong answers about your immigration status is a civic crisis. The difference isn’t the model — it’s the accountability wrapper around the model.
Six rules we’ve committed to
- Cite every answer. No citation, no reply.
- Never take an irreversible action without explicit human consent.
- Refuse rather than guess. Uncertainty is a feature.
- Offer human escalation at every step with no friction.
- Log every decision to a public audit record.
- Welcome appeals — and default to granting them.
The right question is not «can an AI make this decision?» The right question is «how do we make any decision, human or machine, appealable?»
Samir Al-Rashid, Chief AI Officer
The quarterly audit
We publish independent audits of the Assistant’s accuracy, bias, and escalation rates every three months. They are not promotional documents. The audit firm rotates annually. We intend to stay nervous about this for a long time.