Why Human Oversight Must Be The Cornerstone Of Every Responsible AI Policy In South African Business

Your starting point must be to resist the urge to prompt a free Gen-AI tool to draft your response. The challenge is broader than a single document: establishing a responsible AI environment should be a strategic priority for leadership.

Responsible AI depends on more than a policy mandate, but your policy could be the foundation of a robust, human-centric governance framework that meets your organisation where it is today while shaping a culture for an AI-powered future.

These ten guiding principles will help you steer the necessary conversations to create a meaningful AI policy tailored for your business context.

1. Human in the loop

Start and end with humans. No high-impact or high-stakes decision should be fully autonomous. Whether regarding credit, recruitment, insurance claims, or investment decisions, human involvement is non-negotiable.

Section 71 of the Protection of Personal Information Act (POPIA) provides that data subjects have the right not to be subject to decisions based solely on automated processing. To best mitigate legal and reputational risk, a human reviewer acts as the primary firewall against algorithmic liability where automated logic drifts away from reality.

2. The why test

If you cannot explain how an AI reached an output in plain language, you should not use it for client-facing or business decisions.

South African regulators, such as the Financial Sector Conduct Authority (FSCA), and our courts are increasingly demanding transparency. Relying on ‘what the AI said’ is an insufficient legal defence. You must be able to trace and explain the underlying logic, and if that logic is opaque or nonsensical, it cannot be used.

3. Local remains lekker

AI models used in South Africa must be tested upfront for bias and monitored continuously throughout their use. This must go beyond basic demographics - it should consider whether the model understands South Africa’s languages, cultural nuances, regional differences, and historical context. Many AI tools are trained on Global North data and consequently may be technically powerful but poorly equipped to interpret South African social realities.

This is critical because the Equality Act creates no special exception for AI. Organisations cannot assume that outputs are acceptable simply because a machine produced them. Recent examples of AI-generated propaganda demonstrates how easily these tools weaponise Western symbols, reminding us that AI is never culturally neutral.

4. Rethink IP protection

AI is not only a threat to intellectual property, but an accelerator that, responsibly used, can help refine services and unlock value from existing knowledge. Each organisation must first define what intellectual property (IP) means in its specific context - what it creates, owns, and regards as strategically important.

From there, treat AI as both an enabler and a risk. Provide clear guidance on what information may be fed into AI tools, what must remain protected, and how third-party content (research, data, or images) is utilised. The goal is not for the AI policy to replace an IP strategy, but to ensure AI supports innovation without exposing trade secrets or infringing on third-party material.

5. Transfer data protections

If you are training AI, your data transfers must comply with POPIA’s strict rules regarding cross border movement. Many organisations overlook the fact that most large language models (LLMs) are not hosted in South Africa. The moment local personal information is entered into these tools, cross-border requirements are triggered.

Organisations must know where their AI providers store and access data, and whether those jurisdictions provide adequate protection. Simply put, before using client or employee data in an AI tool, you must understand the data flow and ensure POPIA compliance is baked into the process.

6. Proudly AI

Transparency in AI usage is becoming a regulatory and consumer expectation. Consumers are increasingly demanding that AI-generated content be clearly labelled. Your policy should define when and how AI watermarks or ‘Made by AI’ labels are displayed. Without these disclosures, provocative marketing campaigns could quickly implode due to allegations of misinformation or misleading trade practices.

7. No shadow AI

Shadow AI is when employees use public AI tools to process confidential entity information. While staff may simply be trying to work more efficiently, they can unintentionally expose sensitive or private data.

Banning these tools outright rarely works. It simply pushes usage underground. A better approach is to prohibit free public tools for work-related data while providing employees with approved, enterprise-grade AI tools. These should be centrally governed and configured so that organisational data is never used to train public models.

8. Test algorithmic impacts

Before deployment, organisations require a formal approval process involving the right stakeholders from the outset. This should not be a one-size-fits-all exercise. Low-risk use cases should move quickly, while higher-risk tools require deep review and testing.

A risk tiering process makes AI governance repeatable and defensible. It prevents legal and compliance departments from becoming bottlenecks while creating an evidence trail of decision-making. This trail proves what was assessed and why the organisation’s decisions were reasonable if something later goes wrong.

9. Consider liability

An AI policy shouldn't be a catch-all for every expense, but it must prompt the organisation to consider how AI-related harm is managed within existing risk and insurance frameworks. This includes identifying potential harms, defining escalation paths, and ensuring current indemnity arrangements are adequate to handle potential compensation or damages.

10. Kill switch authority

Your policy must clearly define ownership and escalation responsibilities. It is not enough to name a person in a document. Rather, each tool requires an accountable owner and active monitoring.

Where an AI system behaves unpredictably, breaches limits, or creates material risk, the responsible parties must be empowered to pause or disable it immediately. A practical shutdown process and an approved fallback plan are essential for containment while an issue is investigated.