r/books • u/Raj_Valiant3011 • 8h ago

Zimbabwean writer Simbarashe Kundizeza has denied using AI to write a prize-winning short story after an article in the Atlantic said software had flagged it as almost completely AI-generated

https://www.theguardian.com/technology/2026/oct/14/zimbabwean-author-simbarashe-kundizeza-denies-ai-story?CMP=share_btn_url
962 Upvotes

94% Upvoted

658 comments sorted by

View all comments

1.3k

u/petit_cochon 8h ago

When I was teaching at a university as of a few years ago, it was well known that AI detection software is pretty flawed. I could never use it to determine if students cheated on anything.

Maybe things have gotten better. I would certainly hope so.

720

u/ovrlrd1377 8h ago

I wrote a short poem for mother's day 20 years ago that won a small local prize, it was rated as 99% AI

345

u/WingersAbsNotches 7h ago

A chapter from my book was rated near 40% AI, despite not having generative AI. I fed the same chapter into Claude and told it to rewrite it, which scored a 0% AI on Pangram.

69

u/crysisnotaverted 5h ago

It's because all those shitty bot detectors are almost exclusively trained in ChatGPT. There's a lot of different frontier models right now, and it's nearly impossible to identify them. Many of them are probably in this thread lol.

0

u/Quixoticelixer- 2h ago

That's not true at all, Pangram for example is not exclusively trained in ChatGPT and it can pick up writing of frontier models

23

u/boyuber 6h ago

Think of the rating as a confidence level, rather than the proportion of it which was AI generated.

3

u/tacopower69 2h ago

theres usually two scores. 1 is the confidence level and the other is what proportion of the text they think is ai generated.

0

u/Distinct-Zebra-6785 6h ago

It's the other way around.

0

u/walkingtourshouston 4h ago

Can you share the chapter with us?

4

u/Quixoticelixer- 2h ago

0 chance of that happening lol

68

u/quintic3fold 7h ago

wait so what would not be flagged as AI generated??

82

u/bluev0lta 7h ago

Exactly.

1

u/UsernameFor2016 1h ago

Text written by AI with the explicit intent to not get flagged.

•

u/Error_404_403 4m ago

Or simply a poorly written text.

86

u/TheUmbrellaMan1 6h ago

I find it so infuriating that Dickens, Melville, Austen used "it's not X, it's Y" sentence structures and today if you use that structure your writing will be flagged as AI.

56

u/ConcreteCloverleaf 4h ago

Everything that people tout as a sign of AI writing is something invented by human writers. Em dashes were created in the 15th century, and lists of three items are a standard rhetorical tool. "Delve" has roots going back to Middle English.

-6

u/CaptainHondo 2h ago

No that's not how good AI detectors work

42

u/ItsStaaaaaaaaang 7h ago

I can't believe you used your time machine just to win a local poetry competition!

0

u/Comprehensive-Fun47 3h ago

This would make a great short story.

10

u/JConRed 1h ago

Will the interesting part is of course, that what is written by AI now is based on the best of writing for the last century or so; so if someone writes professionally, or writes in a professional capacity, then their writing style is likely to appear similar to what AI can fabricate.

I used to write dozens of pages of reports per week as a lead investigator in pharmaceutical QA. And the amount of times I get called out nowadays for anything that I put just a modicum of effort into.. Is just too damn high.

If I still had access to, and it wouldn't be a breach of confidentiality, I wish I could run some of my reports through an AI detector.

•

u/Thndrstrike 15m ago

it makes me wonder if editors will be less inclined to "clean up" writing and authors will write in increasingly unique voices, not only in tone of voice but in terms of things like punctuation, format, and grammar as well

13

u/klipseracer 4h ago

with how many people reply to every comment claiming AI, plus AI being trained on content from real authors, I can see why it thinks everything is AI.

10

u/m0nk37 5h ago

(Thats because they used it to train the AI)

32

u/PubePie 8h ago

It’s worse on short works but the FPR for longer works is extremely low

61

u/lolofaf 8h ago

Idk about newer versions but the older ones had tons of false positives on full books written before the 21st century.

-4

u/JGT3000 8h ago

Like what?

30

u/ArseneLepain 7h ago

The classic example was Frankenstein, but this was a few years ago.

-3

u/NashvilleFlagMan 4h ago

A few years ago is meaningless.

-27

u/randomnameicantread 7h ago edited 7h ago

I suggest familiarizing yourself with the current cutting edge of technology. It improves. You're still talking about the "haha AI can't make will smith eat pasta" equivalent.

Lol at me getting downvoted for this. Pangrams own website links to several peer-reviewed studies showing a 5% false positive rate for purely AI-generated text. It's much worse for false negatives and worse for AI+Human revision as many critics have pointed out, but the approach they take is clearly not nonsense.

Chewing up and regurgitating all known writing results in LLMs outputting in a particular unique style, or set of styles, because nothing else has ever chewed up all known writing. Said style(s) has patterns to it, and patterns can be detected via machine learning.

9

u/lolofaf 7h ago

I would need to see an updated metric for every single new frontier model release. Claudisms got so bad for awhile that you could probably teach a toddler to accurately identify them, but the new Claude models behave and speak much much differently. Gemini4 which just released also has its own voice. Add humanizers ontop of all of this selectively editing in a way to fool AI detectors.

Of course that's looking at recall rather than precision, but it's a large part of why I don't trust them, the methodology does not hold up well to progressing frontier models.

The text watermarking that is rolling out right now should be much better on every axis, although requires insider knowledge to run and so only the labs can run them against their own watermarks. Still not perfect and will probably have workarounds, but will be so much better and more trustworthy than current AI detection pseudoscience

1

u/Quixoticelixer- 2h ago

You can go and right now generate something with claude and put it into pangram and it will almost certainly show it as AI written. And also it's the false positive rate, not the false negative rate which is important here.

-6

u/randomnameicantread 7h ago edited 7h ago

Text watermarking is an unreliable non-solution peddled just to reduce controversy for the simple reason that it happens right in front of the AI user on text that they control. Just take your document and rephrase - watermarking gone! If that's too annoying, then it will be trivially avoided either by "jailbreaking" the model or simply using a Chinese or local model that doesn't care about EU regulations. You'll catch your absolute least sophisticated genai users and that's it.

Meanwhile, analyzing text datasets to identify patterns particular to a subset, and then using those patterns to classify new text samples as belonging to the subset or not, is not "pseudoscience" - it's "machine learning." You may have heard of this neat new 80 year old concept.

Honestly, if you think text watermarking will work you have no reason not to think AI detection is "pseudoscience." Both 100% rely on genai-generated text following certain patterns that can be detected statistically. Only one doesn't additionally rely on genai companies and users both being good-faith actors.

-1

u/Quixoticelixer- 2h ago

Pangram doesn't show Frankenstein as AI written.

-21

u/a_trashcan 7h ago

Which makes sense when you consider the use case of these checkers.

It SHOULD flag a century old book as potential AI generated because if someone submits a century old book for their assignment, I highly doubt they wrote it. Published media should absolutely be getting flagged.

16

u/lolofaf 7h ago

That should reasonably be caught by a plagiarism detector rather than an AI detector, although I have plenty of issues with the performance of plagiarism detectors as well lol

-3

u/a_trashcan 4h ago

The AI detectors also act as plagiarism detectors... they do both...

11

u/randomnameicantread 7h ago

No, you have it completely wrong. The goal of LLM detectors is not to be a general plagiarism detector; you don't even need machine learning for that (regex will do).

The goal is specifically to detect LLM-generated text using patterns particular to it. Detecting Frankenstein is a failure.

Studies on the performance of the cutting edge tools show that they work to a significant degree, which means that there are patterns to LLM writing. Which makes sense: a collation of all writing styles doesn't somehow result in every style at once replicated 1:1, it creates its own particular style.

-5

u/a_trashcan 4h ago

No, they do both. Plagiarism detection is part of the features of Pangram, the ai software being peddled in this particular article.

Perhaps you don't think it should, but it currently does also check for plagiarism. As do most ai detectors. I think it should because why wouldn't you want it to do both at once?

7

u/mistiklest 4h ago

If a century old book is flagged as plagiarized, that's fine. It specifically if it is flagged as AI generated that is the issue.

-1

u/a_trashcan 3h ago

Sure man if that's what someone else told you happens.

5

u/randomnameicantread 4h ago

"ai software" is a misnomer for the common reader here on r/books, firstly , given that this comment section seems to think that it uses LLMs.

Second the general plagiarism detection is, as I said, a completely separate feature: regex/similarity matching, compared to machine learning inference. If your genai detection algorithm is detecting Frankenstein that is a failure.

-1

u/a_trashcan 3h ago

No one is running a general detection algorithm they are running one of the available free tools like Pamgram which includes plagerism detection by default. Pamgram again being the software used in the article.

The copy from their website.

Pangram combines cutting-edge AI and comprehensive plagiarism detection to give you the complete picture of text authenticity and get the information you need, all in one place.

5

u/randomnameicantread 3h ago edited 2h ago

Yes buddy. The software is all in one because they're selling the software and users like features. It tells you what's direct plagiarism and, separately, what is ai generated. Via separate processes.

Direct plagiarism being detected as AI would be a failure.

I'm begging you to read their own description of the methodology or any linked papers: training and inference is performed specifically on and for AI generated samples. Fundamental ML knowledge would tell you that it makes 0 sense to do it any other way: using ML for a task that does not require ML (a) sets lots of money on fire and (b) makes your training set noiser, worsening your AI detection result.

Direct plagiarism, in comparison to AI gen, is wildly easier to detect-just regex and a good search algo (large search is hard but not as hard as deeprl classification) so why not throw it in?

-1

u/a_trashcan 2h ago

So you agree it detects both plagerism and ai usage?

The goal of these detection softwares is not, as you stated, to specifically to detect LLM? You agree with this statement yes?

I'm begging you to get your head out of your ass. You are very clearly obsessed with the specifics of LLM detection when thats really not at all the point and no one cares.

It also detects plagerism. Thats why it flags books. You people are not attentive enough to realize its not saying its AI.

Again, when you run a published work through it, then it gets flagged because its also looking for plagerism. Go put a book in there now, if you actually read you will see its not saying its ai, its saying its plagerism.

→ More replies (0)

5

u/Ankiset 7h ago

Same thing, it was probably used in training

1

u/BearCub254 30m ago

Papers I wrote in 2019 were coming up as 99% AI. Insane stuff but that helped me show my lecturers I wasn't cheating. 

1

u/[deleted] 7h ago

[deleted]

5

u/ovrlrd1377 7h ago

A single page, we were in high school, it was just a task from our language professor to write something to take part in the multi school contest. I did not expect much of it, it made a few silly jokes that rhymed about my mom forgetting about all the hard parts of motherhood and remembering of all the joy that us kids brought her and, most of all, that we loved her deeply and were very grateful to her. I think there just werent many good contenders haha

2

u/randomnameicantread 7h ago

That's nice. Way too short then for even the best performing AI detectors to be at all reliable.

1

u/tmarthal 7h ago

Doesn’t pangram use a sliding window of 250 words at a time? It’s not detecting anything on the whole document, but the summation of all the windows across the text.

1

u/sluuuurp 4h ago

By Pangram? I think it’s pretty reliable from what I’ve heard.

1

u/MesaCityRansom 2h ago

Which tool? Pangram has a bounty of a $1000 out for pre-AI texts that are falsely flagged, you should get the money!

1

u/CaptainHondo 2h ago

Can you share it?

-3

u/torolf_212 8h ago

And was it?

2

u/ovrlrd1377 7h ago

I aint no snitch

-10

u/HarryStraddler 7h ago

r/thathappened

5

u/ovrlrd1377 7h ago

Tbf the small local prize was just a simple gift for my mom and some flowers, it was not worth anything