Hi {{firstname|everyone}},
You’d think a global giant like Deloitte would know better. But when even one of the world’s biggest consulting firms lets AI run unsupervised, chaos follows.
A recent Australian government report revealed that Deloitte’s AI tool, built on GPT-4, fabricated references in a major submission. Fake citations. Non-existent sources. And a lot of red faces. The irony? The same consultants who sell “digital transformation strategies” were tripped up by their own digital shortcut.
The promise of AI is immense, but so are the pitfalls when you treat it like a magic box instead of a precision tool. Deloitte’s stumble is a reminder that technology may be smart, but it’s not accountable. Humans still are.
So what does this mean for all of us using AI in an industry where accuracy and trust are non-negotiable? Let's break it down.
When AI Isn’t Supervised, It Can Undermine Everything
Generative AI is built to sound right, not be right. It’s designed to fill gaps confidently, even when those gaps are fiction.
When humans fail to review what AI produces, those fictions get presented as facts. In Deloitte’s case, it wasn’t a single wrong output; it was a breakdown in process and accountability.
For many models, the AI hallucination rate in general knowledge tasks is nearly 9.2%.
For any firm, whether consulting, legal, or accounting, reputation depends on the reliability of information. Once a client spots an error, it raises questions about everything else that’s been delivered. The speed of AI means small mistakes multiply fast.
That’s why firms need review systems in place before any AI-generated work leaves their desk.
Action point: AI should never produce a “final” document. It’s a draft tool, not a deliverable tool. Firms that understand this distinction will avoid reputational damage down the line.
Human Oversight Is Not Optional
AI can automate, summarise, and analyse, but it cannot judge. That’s the human’s job.
According to Infosys, 95% of executives using AI have experienced at least one mishap, yet only 2% of firms meet the responsible AI standards.
A well-governed AI workflow always includes review layers: trained professionals who validate the data, test assumptions, and ensure outputs align with firm standards.
The Deloitte case exposes a governance gap that exists in many large organisations, the assumption that AI tools are plug-and-play. In reality, every AI deployment needs a framework:
Defined use cases (where it’s safe to use AI)
Review protocols (who signs off before it goes public)
Escalation rules (what happens when AI gets it wrong)
Firms that invest time in these controls can safely scale AI adoption. Those that don’t risk embarrassing, and costly, public errors.
Action point: Make AI review part of your workflow, not an afterthought. If no one owns accuracy, everyone is accountable when it fails.
The Fix Lies in Human-Centred AI Governance
AI didn’t hallucinate in a vacuum. In fact, it did so because no one asked why or how the system produced what it did.
Many firms are rushing to deploy AI tools without the human checks and balances that ensure credibility. Models can be trained, but judgement can’t be automated. You still need people, skilled auditors, analysts, and reviewers, to validate the data, question the logic, and sign off on outcomes.
It’s no surprise that only 24% of companies today have a formal AI oversight framework in place (PwC, 2024). The rest are still treating AI as a magic box.
Good AI governance creates trust. It’s what separates a firm that experiments with AI from one that masters it. Governance means:
Setting clear internal guidelines for AI usage.
Creating audit trails for every output and decision.
Training teams not just to use AI, but to question it.
Having escalation processes when the system fails or fabricates.
Action point: Don’t wait for a “perfect” AI system. Build small, controlled pilots. Train your team to use it responsibly. Scale only once the checks work.
How We’re Building It Right at Samera.AI
The Deloitte episode is a reminder that unchecked AI can do more harm than good.
So, while we’re deep into developing our own models, we’re doing it with clear guardrails, data accuracy, explainability, and human oversight, at every stage.
We’re piloting Samera AI within our own firm first, testing, learning, and fine-tuning, before bringing it to the wider accounting world. The goal is to build trustworthy intelligence that firms can rely on without losing control of their judgement.
Book a discovery call to explore how responsible AI can fit into your firm’s future:
Cheers,
Arun
AI Agents That Cut Support Costs By Up To 80%
AI Agents Designed For Complex Customer Support
Maven AGI delivers enterprise-grade AI agents that autonomously resolve up to 93% of support inquiries, integrate with 100+ systems, and go live in days. Faster support. Lower costs. Happier customers. All without adding headcount.