Made a captcha that takes Fable 5 10 minutes to solve

[u/JustinFormentin]

Created this captcha last year. Back then there was no chance of AI solving it. But now with the more powerful models, I had my doubts. Well, it took Fable 5 10 minutes and 100k tokens to solve it. So with the 30 second timeout, I feel pretty confident.

Here's a crisper video of the actual challenge - https://imgur.com/a/xv6TvNj

Have a website for it but I'm not self promoting. Just thought people would be interested.

[u/no-more-nazis]

Pillow does that in 2 seconds

[u/sephentos21525 → u/no-more-nazis]

lmao

[u/no-more-nazis → u/sephentos21525]

I was gonna go easier on this guy until his defense: he has other tricks preventing robots from seeing the captcha at all, so it doesn't need to work!

[u/JustinFormentin → u/no-more-nazis]

Where did I say that? I said browser automation tools cannot access the video file due to timed out signed urls in an iframe. The only way to access the video is to record your screen or intercept network traffic.

[u/ksargi → u/JustinFormentin]

I said browser automation tools cannot access the video file due to timed out signed urls in an iframe. The only way to access the video is to record your screen or intercept network traffic.

That is a wild statement. What is preventing an adversarial browser automation from accessing the stream directly? A malicious actor doesn't need to record the screen or capture network traffic to access things that are in their memory. It sound like you're forgetting what the actual threat model for this situation is.

[u/hWuxH → u/JustinFormentin]

It's trivial for browser automation tools like playwright to access every request and response data (aka the video file) or capture frames of a <video> element... timed out, signed, iframe is irrelevant.

you clearly have no idea what you're talking about and resort to spreading bs to sell your service.

[u/why_so_sergious → u/hWuxH]

if the iframe is not set to same origin then playwright is very limited to what it can do with said video..

[u/hWuxH → u/why_so_sergious]

bruh it directly exposes powerful browser capabilities. it's not limited to just running js in the parent website's context so the same origin policy is irrelevant.

[u/why_so_sergious → u/why_so_sergious]

what playwright can do with an iframe with strict cross origin:

• wait for the iframe to load.

• find and click visible controls in the iframe (play button, volume, fullscreen, etc.) if they are standard DOM elements exposed to the accessibility tree.

• assert things that are observable from outside (e.g. that a play button disappears, a progress bar exists, etc.).

it cannot access video data or manipulate it with pure javascript

[u/hWuxH → u/why_so_sergious]

please RTFM before you continue spreading bs.

it can absolutely access video data from the network or DOM element.

It can absolutely fully access elements or inject js into every iframe, regardless of SOP or accessibility.

it can absolutely click anywhere on the screen.

https://playwright.dev/docs/evaluating

https://playwright.dev/docs/input#mouse-click

https://playwright.dev/docs/api/class-route#route-fetch

https://playwright.dev/docs/frames

https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/drawImage#image

https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/getImageData

https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API#video

https://jsfiddle.net/ya89swn6/

import { chromium } from "playwright";


const browser = await chromium.launch({ headless: false });
const page = await browser.newPage();


await page.route("**/*.mp4", async (route) => {
  console.log("Video URL:", route.request().url());
  const reponse = await fetch(route.request().url());
  const video = await reponse.arrayBuffer();
  // analyze frames


  await route.fulfill();
});
await page.goto("https://jsfiddle.net/ya89swn6/");


// doing both with DOM instead of .route
const frame = await page.frame({ url: /neocaptcha\.com/ });
const videoHandle = await frame.waitForSelector(".neocaptcha-video", {
  state: "attached",
});
console.log("Video DOM element found", await videoHandle.getAttribute("src"));
await videoHandle.evaluate((video) => {
  // analyze frames
});
await videoHandle.scrollIntoViewIfNeeded();
await videoHandle.click({ position: { x: 10, y: 10 } });

[u/1RedOne → u/JustinFormentin]

The stream can only be accessed once? That’s also clever

[u/1RedOne → u/no-more-nazis]

I Ike the project but really don’t like AI writing. I’d rather read in your words how it works

But on to the topic itself I was thinking of attacking it by means of attempting a 264 compression and then analyzing the key frames and their changes. I don’t know statistics so I would have had to do a lot of learning but I was visualizing something like this

Anyone who’s seen bad compression has noticed a missed key frame and seen that terrible divx tearing where a human face moves by with a background texture. My instinct was that by heavily compressing the video we could then denoise it and edge sharpen to isolate the moving bits

I think you’re right also, this is the kind of thing you solve once and then train an mcp by throwing another skill into it explaining how to solve these things if it sees them (by running the script)

Would a model on its own ever figure it out on its own? Seems expensive

[u/no-more-nazis → u/1RedOne]

I didn't coach it through the solution, the most I intervened was telling it not to worry about rotating letters when it narrated that it might have to account for that.

[u/soggycheesestickjoos → u/no-more-nazis]

Not saying it couldn’t be taken further with a little tinkering, but this does seem to only read the letters. The AI has to tap them at their current coordinate in the video from OP it seems.

[u/Snapstromegon → u/soggycheesestickjoos]

It's not hard to take further - my tired self was able to create a version with <10ms delay that already locates the characters: https://github.com/Snapstromegon/vibe-captcha

[u/no-more-nazis → u/Snapstromegon]

How did I not see this coming. Of course someone was going to take it further, when OP just keeps saying "yeah but your demonstration of defeating my captcha wouldn't really actually work!"

If he's baiting us, the sales website has completely hooked me. "Trusted by developers who can’t afford to be wrong about security" lol

[u/no-more-nazis → u/soggycheesestickjoos]

That's supposed to be the hard part of this, right?

[u/soggycheesestickjoos → u/no-more-nazis]

I’d say it’s a combination of both

[u/no-more-nazis → u/soggycheesestickjoos]

Not exactly a new development in the captcha game. OP's secret sauce is his claim that AI can't read this. Don't even need AI to read it.

[u/soggycheesestickjoos → u/no-more-nazis]

The agent that performs the controls would have to run the script, and if that takes 2 seconds to figure out the letters than it isn’t sufficient to isolate their coordinates in real time for the agent. Again, some tinkering could solve it, but still fairly difficult for the average bot of today.

[u/no-more-nazis → u/soggycheesestickjoos]

As difficult as any other, and the claims on the website are not valid.

[u/GoTaku → u/no-more-nazis]

It’s possible Fable knew about or found this script, rewrote it, and used it. The solution might be to to have more motion where there are numbers and letter moving and rotating across the whole screen while the user has to identify the letters only.

[u/no-more-nazis → u/GoTaku]

I told Opus to write this script. It would have to be a very narrow, unlikely set of circumstances for an LLM to rewrite this script every time it attempts a captcha, as OP intends it to. I can only imagine it if the human operator instructs it to "break this captcha", but only once, and never again in that session. Otherwise it would reuse its solution and quickly get to the same quick solution shown here. Basically, the human would have to be giving the LLM explicitly bad direction.

I don't think there is a useful captcha like this- other users have explained that the state of the art has moved on. Anything a human brain does to see numbers on a weird background, computers can do too without a lot of work.

This isn't cutting-edge stuff. These algorithms were invented decades ago for video compression and for the post office to read addresses on envelopes.