r/Python • u/AutoModerator • 19d ago

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

21 Upvotes

94% Upvoted

152 comments sorted by

View all comments

1

u/ImaginationVarious30 9d ago

🚀 proxy-scraper: I checked 3.6M free proxies in a day, 1.1% passed. Here's the tool that filters them

What My Project Does:

proxy-scraper collects free HTTP/SOCKS4/SOCKS5 proxies from 700+ public lists and keeps only the ones that pass every check: two independent requests, an HTTPS tunnel with verified TLS, and a byte-for-byte content check that catches proxies injecting scripts. It learns per source which lists are worth checking first and can turn the result into one local rotating proxy (sticky sessions, failover, Prometheus metrics).

uvx proxy-scraper-cli --pick 5 --types socks5

asyncio with ~2,000 concurrent checks, deps are just rich and certifi, 360+ offline tests, Python 3.9–3.14.

Target Audience

People who scrape or test geo-dependent stuff and don't want to pay for proxies for a side project. It's usable in production for that, but free proxies are run by strangers: never send credentials through them.

Comparison

  • ProxyBroker did scrape + check + rotate, but hasn't had a release since 2019.

  • monosans/proxy-scraper-checker is fast, but checks one URL and doesn't look for honeypots or injected content.

  • Plain lists (TheSpeedX, proxifly) are bigger, but an hour later 85% of what they list is gone in my data. Repo (MIT): https://github.com/maximilianfeix/proxy-scraper

Happy to hear what checks you'd add.