AI Detectors Lose Edge as Models Mimic Human Style, Says Dr Mario Landman

Dr Mario Landman of the Academic Centre of Excellence for The IIE and Advtech, says the primary reason for the retreat from AI detection is a lack of accuracy.

“AI detectors do not know if a machine wrote a text; instead, they measure statistical signatures like ‘perplexity’ (how predictable the language is) and ‘burstiness’ (variation in sentence rhythm). As generative models have evolved to mimic human style more effectively, these signatures have become blurred,” he says.

Independent evaluations show that while some tools claim 99% accuracy, their effectiveness drops to between 60% and 80% as soon as a student manually edits or adds "humanise" when prompting the AI.

Furthermore, newer models like Claude 3 generate natural-sounding prose that frequently evades mainstream checkers.

For many administrators, using such probabilistic tools to make life-altering disciplinary decisions is becoming an unacceptable risk to due process.

The crisis of fairness and bias

For South African institutions, the most damaging aspect of AI detection is documented bias against non-native English speakers, notes Dr Landman.

“Research has shown that detectors disproportionately flag ESL (English as a Second Language) students because their writing often uses more formal, standardised structures that the software mistakes for machine-generated patterns.”

One landmark study found a 61.3% false positive rate for TOEFL essays written by Chinese students, compared to just 5.1% for native speakers. In a multilingual country like South Africa, where English is often a second or third language, relying on these tools creates a systemic equity crisis that risks unfairly penalising students from disadvantaged backgrounds.

The devil’s bargain of efficiency

Dr Landman says the complexity is deepened by what scholars call a "devil’s bargain" in modern academia.

“AI can automate lesson planning for lecturers and generate plausible essays for students, creating an appearance of productivity while hollowing out actual learning. This leads to the rise of ‘shallow knowledge workers’ - graduates who are proficient in prompt manipulation but deficient in critical analysis and independent reflection.

“By switching off AI checkers, universities are forced to confront this erosion of cognitive capacity. Rather than attempting to detect the machine, they are redesigning the work to make human thinking visible.”

The way forward: A mixed ecology of assessment

The emerging way forward in South African higher education is a shift from "policing" to "stewardship", says Dr Landman. He says the focus is moving toward:

• Authentic assessment: Moving away from take-home essays toward oral defences (viva voce), in-class writing benchmarks, and practical demonstrations.
• AI-integrated assessment frameworks: Implementing clear frameworks that define acceptable AI use across different assessment contexts. This may include a tiered approach where AI use is prohibited, permitted for specific parts of an assessment, or fully integrated into the assessment process without penalty, provided its use is transparent and aligned with the learning outcomes.

• Process-based grading: Grading the "learning journey" by requiring students to submit research logs, drafts, and "Epistemic Meta-Reflections" where they justify their interaction with AI.
• Human-in-the-loop frameworks: Implementing automated grading only when it includes mandatory human review to ensure factual accuracy and fairness.
• Transparency and disclosure: Replacing bans with disclosure requirements, where students must cite which tools they used and for what purpose.

As South Africa finalises its Draft National AI Policy – which ironically ran into an early roadblock after it was found the first iteration was drafted by AI - the higher education sector has an opportunity to ground AI governance in the philosophy of Ubuntu, with its emphasis on interdependence, human dignity, and collective responsibility, says Dr Landman.

“The goal should not be to win an unwinnable technological race, but to establish a renewed contract of trust: one in which AI is used as a scaffold for thought, not a substitute for it.”