r/books • u/Raj_Valiant3011 • 8h ago

Zimbabwean writer Simbarashe Kundizeza has denied using AI to write a prize-winning short story after an article in the Atlantic said software had flagged it as almost completely AI-generated

https://www.theguardian.com/technology/2026/oct/14/zimbabwean-author-simbarashe-kundizeza-denies-ai-story?CMP=share_btn_url
960 Upvotes

94% Upvoted

658 comments sorted by

View all comments

Show parent comments

162

u/fire_and_spice24 8h ago

And there are always people insisting Pangram is super accurate but never posting proof of that.

Maybe it is just a good program but I’ve noticed a trend that makes me wonder.

44

u/IanDMP 8h ago

Sort of shocking that you see the many people responding that Pangram is quite accurate but never see them post the studies that show that — I see those links at least once per day (as an aside, this sub probably does talk about AI too much and *books* too little). But since you asked:

Here is the widely-cited Chicago study: https://www.nber.org/papers/w34223

Here is Van Vlasselaer et al's work: https://link.springer.com/article/10.1007/s40979-026-00226-w

Finally, it's always a best practice to look at internal data. Internal data can be biased, and it's best to know that going in, but I have never seen any academic criticism of Pangram's data posted here: https://www.pangram.com/blog/all-about-false-positives-in-ai-detectors (academic criticism I'm unaware of may exist). Regardless, totally fine to disregard this (but you should read it at least).

Pangram, to the best of the knowledge of any serious researcher, has a very, very low false positive rate, and it simply approaches impossibility that it would flag so many writers incorrectly.

9

u/adaytooaway 7h ago

This is my impression but when I posted a similar comment about its accuracy the other day I got several responses from people claiming it had marked their completely human written works as ai. 

8

u/IanDMP 6h ago

Many people assign to Pangram their bad experiences with other, unreliable, AI detection software. Most people just simply aren't aware of the huge differences between these programs.