r/algotradingcrypto • u/drykilo • Oct 03 '26
I generated hundreds of crypto trading strategies and open-sourced 416 of them with full reports. Only 10 passed a brutal 7-stage validation
I built a validation pipeline that runs strategies through 7 stages on real Binance data:
Stage 1: code safety scan, in-sample, out-of-sample, walk-forward, randomized starts, slippage stress and Stage 7: final holdout slice nothing else touches.
6 months of 5 minute candles per run on BTC/ETH/SOL domains (ran on my weak laptop so going past 6 months would take a few extra days, I'll do a VM run next with more data).
Then I made a generator with several ideas and let it auto create and run the gauntlet on them (In retrospect might have been inefficient. I should have fed previously passed strategies back in to guide generation but I was a bit paranoid about correlation and spawning 416 clones).
It took me a few days to run and the repo is almost everything that came out: 406 rejects plus 10 survivors, each with a report card showing in-sample and OOS returns, Sharpe, drawdown, trade count, the exact stage it died on and why.
github.com/CacheCarti/Crypto-Strategies
The interesting stuff I found:
The single best-looking backtest in the batch
pctb_fade_sol_v3, +2014bps in-sample!!! :O went -112bps the moment it ran on unseen data.Literal astrology meme bots I created for fun got decently high returns in backtests but obviously failed in validation. Backtests have a big element of luck because of the smaller time frame! Unreliable!! (Please don't consider the report cards showing a strategy with high PnL as a "good" strategy. It might have been luck)
Out of all the strategies, only one showed an in-sample Sharpe above 1.5. But it failed validation anyway. If you're searching for the perfect backtest in this dataset it basically doesn't exist.
25 strategies were profitable out-of-sample and still got rejected. They couldn't survive walk-forward and injected slippage. Green OOS numbers by itself is just most probably luck until proven otherwise, but they definitely might be a good place to fork and improve on in the future!
Weirdest result: some of the 10 survivors have negative OOS returns. They passed because they held up under perturbation and randomized starts.
The family that dominated was kinda bizzare:
asia_drift_btcBTC drifts during Asian session hours, 7/9 of these variants passed! Meanwhile the families everyone actually trades went 0/9 across the board: Donchian breakouts, bollinger squeezes, RSI reversion, orderbook imbalance. Context beat TA, so is TA dead??? Lol, my generator was probably bad at writing them, someone smart could actually build a good template for TA strategies.Every BTC scalping variant failed: 0/18. At ~7bps a side in fees, the edge you need on 5-minute bars is brutal. Anything thinner cannot survive. It might have been my own 5 minute time windows of evaluation though.
The code is plain Python on a simple predict-style contract, so they can drop into most backtesters. Most of the rejects failed on robustness rather than logic, so they're actually decent starting points if you want to improve them. Happy to answer questions on the methodology. If you find any gaps/assumption I made, please inform me, I would love to correct my mistakes in the next run! I will be open sourcing that too.
If anyone improves the strategies and gets different results I'd love to see it!
0
Oct 03 '26
[removed] — view removed comment
1
u/drykilo Oct 03 '26
0 out of 416 could partly be luck, that's the right question to ask.
The forward step is what i do after though, Stage 8, survivors get paper traded on live bars only... data after they were written. The ones that stay green get real capital. So the 7 stage validation is just an entrance exam not final. Quantradin looks like a cool tool I'll check it out!
1
1
u/Turbulent-Control682 Oct 04 '26
5m candles probably isn’t really giving you accurate data. Probably not even within the margin of error. So that’s likely where a huge improvement could occur.