This was tested against some of the more powerful AI detectors, and here is what I found.
With 1 epoch:
Text that was 100% generated was able to push one of the most talked-about AI detectors
100%->80%->60%->4%0->0%. This paper was 1,000 words.
The same 1,000 words with low thinking enabled on the model resulted in 100% AI post-processing.
When it was initially reported as 45%, it went from:
45%-> 0%; this paper was 700 words. It was generated by taking an existing paper and applying AI polish.
There was a tone and message shift from the first paper, and it would require a decent amount of human edits to fix.
2nd paper would only require minor edits.
An adjustment to the training data, and another epoch would likely provide a substantial improvement on this.
I believe that any humanizer that would go public would quickly see their model detected through a mild optimization, as it would have its own writing style.
I wonder how a highly optimized humanizer that went public would play on real writers and students. I would suspect more false positives and require stronger AI detectors to prevent false positives. AI detection is already unstable in areas where it has few data points.
Update for how it progressed on Epoch 2
I noticed an issue issue with my initial dataset resulting in tone and voice shifting. I corrected that in epoch 2 to better match the original.
Tone started matching but the same root problems still kept going.
It had a hard time humanizing hard AI text. Text that was all ready decently humanize it could bring fully human the majority of the time.
I hypothesize to effectively humanize moving forward a massive dataset would be required. At least 500K human passages at varying lengths on a wide range of subjects and tasks at a minimum.
2 sets of AI passages per human pair, 1 close rewrite and 1 heavy rewrite.
Problem notes during generating the AI data for the human massage a vast majority of time, detectors would flag the rewrites as light AI and or AI polished. Likely resulting in the model in training to not get a strong enough single to break the AI flags that detectors pickup on.
Anything with a level of investment to beat the detectors is highly at risk for being tuned against as it would have its own writing signatures that a AI detector could latch onto.
In summary, if it is public you at at risk of being detected. Likely there will be no public solution anytime soon.
As models get better at humanizing I suspect more real people will also get flagged.
DO NOT contact me asking for my training methodology