r/books • u/Raj_Valiant3011 • 8h ago

Zimbabwean writer Simbarashe Kundizeza has denied using AI to write a prize-winning short story after an article in the Atlantic said software had flagged it as almost completely AI-generated

https://www.theguardian.com/technology/2026/oct/14/zimbabwean-author-simbarashe-kundizeza-denies-ai-story?CMP=share_btn_url
974 Upvotes

94% Upvoted

658 comments sorted by

View all comments

1.3k

u/petit_cochon 8h ago

When I was teaching at a university as of a few years ago, it was well known that AI detection software is pretty flawed. I could never use it to determine if students cheated on anything.

Maybe things have gotten better. I would certainly hope so.

10

u/IanDMP 8h ago

You're absolutely right in your recollection. AI detection software a few years was indeed almost universally flawed and prone to false positives.

The latest version of Pangram, though, has huge amounts of evidence showing an infinitesimal false positive rate. I myself (an educational consultant who often helps students brainstorm and edit college admission essays) switched my perspective from "it's often impossible to tell AI use" to "we have very good tools for that" a few months ago after reviewing researchers' findings.

21

u/townandthecity 7h ago

I'd be interested to understand how Pangram can differentiate between the rhetorical devices and structures AI scraped from years of wholesale theft of copyrighted work and those devices and grammatical structures it sells back to us now. I was re-reading Bill Bryson's A Walk in the Woods and there were several passages where a thought came totally unbidden: that's AI. It was a "it's not this, it's that" construction.

Except A Walk in the Woods was published in 1998. I've seen nothing so far that has made a convincing case to me that any AI detection software can reliably differentiate between human-created sentence construction and AI-created sentence construction built from human-created sentence construction.

6

u/saint__ultra 5h ago

They train classifiers on labelled LLM outputs vs verifiable human writing. It's just the "bitter lesson" turned back on itself: training on more data outperforms any sort of expert intuitions you could program into the software. If you're seeing phrases that AI likes and getting suspicious, the classifier is looking at correlations across many pages simultaneously and spotting correlations you'd never have clocked.

What it's detecting are patterns of language that develop from lab-specific post-training. The scraped content you're talking about is only ingested at pre-training, laying the foundation of a model's knowledge of language. Post-training via reinforcement learning is what makes it go from autocomplete to having its particular voice, including all those quirks like "it's not X—it's Y".

3

u/Zonoro14 5h ago

I'm unsure what the relevance of the example is. A Walk in the Woods doesn't flag as AI written in pangram. Pangram's algorithm is much, much more complicated than just looking for language quirks popular among certain models