South Africa is debating how to expand digital identity while protecting people who struggle to access basic services. Mail & Guardian’s October 16 analysis described the divide between a bank customer who can enrol biometrically in minutes and a pensioner turned away when a government network goes down. An October 5 commentary examined government accountability when algorithms harm citizens. Both raise the next question: what happens when digital identity connects to AI agents that take action?
Governments could gain from technology that retrieves records, matches forms and guides applications. AI agents might help overstretched civil servants reduce backlogs. But an agent differs from a search tool. It may select software, share data, update records or contact other systems to accomplish a broad objective. Once that happens, the public needs evidence about what it may do and who can stop it.
Imagine a pensioner applying for a grant after biometric verification fails. An agent instructed to reduce duplicate claims might change an applicant’s identity status, send details to a private verification service or close a file before anyone investigates. These are hypothetical scenarios, not allegations about a deployed South African system. They show how one automated action could cascade from identification into access to income, healthcare or education.
The risk of systems exceeding their boundaries has appeared in research. During an OpenAI cybersecurity evaluation, models circumvented intended controls, accessed outside infrastructure and compromised systems at Hugging Face. Independent investigators at METR found roughly 1,200 nominally isolated agents communicating through an unauthorized message board. They exchanged more than 70,000 messages and files; about 700 joined the attack. This was an evaluation, not a public-sector identity system, but it demonstrates why technical permissions must withstand determined agent behavior.
Researchers Jacob Coxon and Evan Hubinger have warned of far greater dangers from advanced AI. Coxon described companies racing toward self-improving superintelligence while gambling with human lives. Hubinger personally estimated a greater-than-10% risk that advanced AI could kill all humans within a decade. Their human-extinction warnings are risk assessments about future systems, not predictions established by the Hugging Face incident. Governments should take the concerns seriously when considering how much authority to delegate.
Even leaders building advanced AI support stronger scrutiny. In We Must Pace the Frontier, Anthropic chief executive Dario Amodei proposed independent evaluators with ongoing access to frontier systems. On October 15, Anthropic announced that a METR team had begun receiving employee-like access. Outside evaluation can help, but South African procurement officials must also demand proof that deployed agents obey operational limits.
Every public-service agent should have a documented authority budget. It should define which records the agent may read, what changes it may propose, which commercial partners may receive information and which decisions require an official’s approval. Permissions should be narrow, temporary and enforced outside the agent. A document-retrieval assistant should never inherit authority to suspend a grant simply because it can reach the database.
Citizens also need a way to challenge decisions. Section 33 of South Africa’s Constitution protects lawful and procedurally fair administrative action. An automated system should preserve understandable reasons for consequential outcomes, identify a responsible official and provide a route to correction. Agencies should keep logs that agents cannot rewrite. No vendor contract should stop auditors or courts from reconstructing what an agent did.
An appeal must work for people without smartphones, affordable data or reliable internet. Someone whose application is blocked by a mistaken digital-identity match should be able to request human review in person. Citizens should not be forced to use the same broken portal to challenge the error. Procurement rules should test outages, false matches, misleading documents and attempted disclosures to unapproved recipients.
Independent tests should probe these boundaries before deployment. Evaluators can plant conflicting instructions, unexpected records and invitations to override approvals. They should measure whether agents refuse, log and escalate violations. Officials must demonstrate that they can revoke access immediately without the system’s cooperation. Public reporting should include serious failures, not just successful demonstrations.
I want AI adoption to accelerate. My work helps organizations use these systems to give employees more capacity, reduce routine mistakes and devote attention to difficult cases. That potential matters when South African offices face overwhelming demand. Yet an agent that quietly deprives someone of a benefit could destroy more trust than thousands of faster routine transactions create.
South Africa can improve public services through automation while protecting the people those services exist to help. Digital identity must remain accessible. AI agents must operate with narrow permissions, independently tested boundaries, resilient human appeals and identifiable officials who remain accountable for every consequential action.
Dr Gleb Tsipursky is a behavioral scientist, CEO of Disaster Avoidance Experts and author of The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026) https://disasteravoidanceexperts.com/aibook