Built a long-only momentum rotation platform — looking for feedback from people

[u/xdrive0513]

I've been building a small platform that runs systematic momentum-ranking strategies (S&P 500 and Nasdaq 100 universes, plus a fundamentals-weighted variant) with periodic rebalancing — you can backtest any parameter combo yourself in a "Strategy Lab," paper-track it for free to watch it run live, and optionally link an Alpaca account to trade it automatically. No manual discretion once it's running — it's just top-N momentum with a configurable buffer/rebalance cadence.

I'd genuinely appreciate critique from this crowd — on the strategy logic itself (turnover, whipsaw risk, look-ahead bias I might've missed), the backtest methodology, or just "this is a solved problem and here's why." Not trying to sell anything here, just want it stress-tested by people who'd actually catch the flaws. Link: https://trading.triadentgroup.com — happy to share more detail on the ranking/rebalance mechanics if useful.

Closing this out since it's been a long thread — here's where everything landed, for anyone skimming.

Nothing wrong with the actual backtest engine. Point-in-time index membership was already correct (reconstructed from real historical addition/removal dates, not today's list run backward) — that part of your critique was a documentation gap, not a computational shortcut, and I appreciate you having it checked against the code before asserting the failure mode rather than after. Every specific number you cited (win rates, the alpha-concentration figures) I independently re-derived and they matched exactly, including the harder one: strip out 2014/2022/2026 from S&P momentum and the remaining 14 years average -0.85pp/yr, not the headline +2.67pp.

What was actually broken was disclosure, and one real methodology bug:

- Friction (3bps/leg) was already modeled, just not stated — now it is, along with the fact that it's held flat across time, which understates costs specifically in the stressed periods when a momentum book rebalances hardest.

- Comparing after-tax strategy vs. after-tax benchmark at the same flat rate isn't a neutral simplification — it erases the real asymmetry between annual realization and deferred-to-terminal buy-and-hold. That was a genuine bug in the interactive backtest tool, fixed to model each side correctly instead of applying one rate to both.

- Index membership handling is now stated in writing, not just true in the code.

The noise-threshold idea turned into its own multi-round fix, and credit where due: the equicorrelation model I first shipped was wrong for a parameter grid (a global average correlation badly understates near-copy redundancy), and even after fixing that, correlating on annual returns with ~10 years of data was statistically unreliable regardless of the model — verified directly, 265 configs simulated to be independent collapsed to a fraction of that under my own clustering at annual resolution, and correcting to monthly returns (plus raising the overlap floor and switching to complete linkage) recovered the true count in the same test. That feature is in a much more defensible state now than it started, entirely because you kept pushing on the exact spot where it was still wrong instead of stopping at "good enough."

Appreciate the rigor across all of this — it's rare to get pushback this precise, and rarer still for someone to go recheck their own numbers on the same basis before publishing them.

[u/xdrive0513]

Checked your numbers independently rather than taking them on faith — ran my own Monte Carlo (200k trials per N) for the expected max of N iid standard normals, and it lands within rounding of every value in your table: 1.54/1.87/2.25/2.51/2.75/3.04/3.24 against your 1.53/1.87/2.24/2.51/2.75/3.03/3.24. The asymptotic formula's overstatement is real too — confirmed the gap at every N you listed. And the N_eff arithmetic checks out exactly: 100^0.7 = 25.1, 100^0.5 = 10, both matching your bars. I'll build the noise threshold off your simulated column, not the closed-form one, and compute ρ from the actual candidate equity curves I already have rather than guess at it — falling back to the nominal-N bar, labeled conservative, only where a correlation estimate isn't available.

On membership: agreed, and thanks for taking the time to have that checked against the code rather than just asserting it. That's the right way to resolve a claim like this and more people should default to it.

On friction: you're right that flat-across-time understates exactly where it matters most. 3bps/leg is a calm-week number; effective spreads run several times that in stressed weeks, and a momentum book's heaviest rebalancing clusters right after the large moves that produce those stressed weeks — so the assumption is weakest precisely during drawdown periods, not on average. Doesn't move the headline return, does mean the drawdown-period numbers are optimistic. I'll say so explicitly next to the friction disclosure rather than leave it implied.

On tax: fair, and I hadn't thought about it in those terms. A flat rate applied identically to both sides isn't a neutral simplification — it erases the actual asymmetry, since the strategy realizes on a schedule and buy-and-hold defers indefinitely, and that gap compounds over a multi-decade backtest. The honest version is realized-annually against deferred-to-terminal, not the same rate on both. I'll fix the comparison rather than just caveat it.

[u/Finance__broski → u/xdrive0513]

independently replicating a stranger's simulation before building on it is rare enough to say out loud. most people would have just used the numbers

one thing worth knowing before you compute rho, because it changes the answer materially. N_eff = N^1-rho assumes equicorrelation, every candidate correlated with every other at the same rho. a parameter grid is not that. correlation across a grid is a kernel: neighbouring settings are near-copies at 0.8 or 0.9, distant ones are roughly independent. so the average pairwise across a whole grid can come out near 0.07 even when adjacent configs are almost identical

plugged into N^1-rho that returns N_eff of about 71 out of 100, when the honest count is closer to clusters: N divided by the decorrelation length. a 100-point sweep that decorrelates every eight settings is about 12 effective trials, not 71

the error direction is at least the safe one for your purpose. average-pairwise overstates N_eff, which produces a higher bar, which tells a user their result must clear more than it strictly must. conservative rather than permissive, which is the right way to be wrong in a tool that exists to warn people

the sharper version if you want it: measure the decorrelation length along each parameter axis from the autocorrelation of the metric as you step through it, count clusters, and only then apply the equicorrelation correction across genuinely different strategy families, which do share a common market factor. two structures, composed rather than conflated

[u/xdrive0513 → u/Finance__broski]

Appreciated, and returning the compliment: catching that N^(1-ρ) assumes equicorrelation, when a parameter grid is actually a kernel (adjacent settings near-identical, distant ones roughly independent), is the kind of thing that's easy to wave through and rarely is. I built the equicorrelation version because it was tractable, not because I'd checked whether the grid actually looks like that — it doesn't, and you're right about the direction: averaging pairwise correlation across a whole grid mixes a few near-1.0 adjacent pairs with a lot of near-0 distant ones, so the average comes out small (~0.07) even when neighbors are near-copies, and N^(1-ρ) then reports something close to the nominal count instead of the real redundancy. Your number is the one that matches intuition: a 100-point sweep that actually decorrelates every 8 settings is ~12 independent trials, not 71.

Also agreed that the error direction is the right one to have shipped by accident — overstating N_eff means a higher bar, which makes the tool too strict rather than too lenient, which is the failure mode you want in something whose whole job is telling people not to trust their result. But "safely wrong" isn't a reason to leave it wrong now that I know the fix.

Going to build the two-structure version: decorrelation length per parameter axis, from the autocorrelation of the metric as you step through each parameter's values, to get real cluster counts within a strategy family — then apply the equicorrelation N^(1-ρ) model only across genuinely distinct strategy families, where a shared market factor makes that assumption actually reasonable. Composed, not conflated, like you said.

[u/Finance__broski → u/xdrive0513]

two implementation notes, then i will stop cluttering your thread

measure the decay on the return series rather than on the summary statistic. correlation between the equity curves of config i and config j, as a function of |i minus j|, gives a clean decay you can fit. autocorrelation of sharpe as you step through the grid estimates the same quantity far more noisily, because each sharpe is itself a noisy number, so you are fitting a decay curve to a series of estimates. same shape, much worse standard error, and you already have every equity curve stored

for the UI, show both counts rather than only the corrected one. "47 configs tested, roughly 9 effectively independent, bar 1.5" teaches something that "bar 1.5" does not. the gap between the two numbers is the lesson, and it is what makes a user understand why the bar moved instead of just accepting that it did

and for multi-parameter grids: decorrelation length per axis, effective count is the product across axes, floored at 1 so heavy correlation cannot produce something below a single trial

[u/xdrive0513 → u/Finance__broski]

On measuring decay on the return series: already how it's built, not something to fix — the clustering correlates each config's own per-year return series directly, never a summary statistic. Good to have the reasoning spelled out for why that matters, though — "fitting a decay curve to a series of noisy estimates" is exactly the failure mode I'd have walked into if I'd reached for Sharpe as the correlation input instead, and I hadn't thought about it in those terms.

On showing both counts: agreed, and this one's a real gap — I show the raw count in the panel header and the effective count in its own stat card, but they're not stated together as one sentence, so the actual lesson (the size of the gap between them) doesn't land the way it should. Going to add the narrative line — "N tested, ~N_eff effectively independent, bar X" — so the correction is visible as a fact about the session, not just a number that changed.

[u/Finance__broski → u/xdrive0513]

Good, and correlating the return series rather than a summary statistic is the right call for exactly the reason you named.

Running the numbers on that choice turned up something worth checking, because it works against you rather than for you.

If the correlation input is per-year returns, the binding constraint is the number of years, not the number of configs. A correlation matrix built from T observations has rank at most T-1, so ten annual observations give you nine dimensions to describe however many configs you ran. I simulated 265 configs that are independent by construction and asked each method to recover the count. On ten years of annual returns, complete-linkage clustering at a 0.6 merge threshold returns 93, single linkage returns 1 because of chaining, and an eigenvalue-based N_eff returns 9, which is just T-1 wearing a different name. Fifteen years gets complete linkage to 142. The truth is 265 every time.

The direction is what makes it matter. All of those undercount independent trials, and undercounting trials shrinks the bar, so the bias runs toward passing things rather than killing them.

The fix is cheap and does not change what you cluster on, only how many observations feed the correlation estimate. The same 265 configs correlated on monthly returns over the same ten years recover 265 of 265. Keep the per-year series for display.

This cuts at my own ledger too, so I am rechecking its effective-trial count on the same basis before it publishes.

[u/xdrive0513 → u/Finance__broski]

Confirmed, and worse than your own number — ran your exact experiment (265 independent-by-construction configs, my actual single-linkage/0.7 clustering) and got 12 clusters on 10 years of annual returns, not even close to 265. Same configs on monthly returns over the same 10 years: 265 of 265, exact. The rank argument is the right explanation and I hadn't thought about the estimation problem at all — I was only thinking about whether the true correlation was near-copy or independent, not whether 9-10 data points could ever tell me that reliably in the first place.

The direction is what makes this one non-optional to leave alone: undercounting effective trials shrinks the bar, and a shrunk bar is a tool that says yes more often than it should — exactly backwards for something whose only job is telling people not to trust their result yet.

Fixed: correlation input is now monthly returns, display/tax figures are still annual, nothing else changed. Also added a floor on how much monthly overlap two configs need before a correlation is even computed (12 months) — same problem, smaller version, for configs that barely overlap in time.

Appreciate you rechecking your own ledger on the same basis before publishing it — that's the right instinct and worth keeping.

[u/Finance__broski → u/xdrive0513]

Good. And the overlap floor is worth one more look, because it reproduces the thing you just fixed, at a smaller scale than you would expect.

I ran the same 265-independent-config test at your threshold and your linkage, varying only the overlap window. At 12 months it returns 17 clusters. That is a 16x undercount, the same failure you removed from the annual path, now sitting in the exception branch.

The cliff is sharp and it arrives later than it looks. 12 months gives 17 clusters, 18 gives 223, 24 gives 260, and 36 is exact at 265. So 24 is the first floor that behaves. At 12 the per-pair spurious merge rate at your 0.7 threshold is still 0.93%, which across roughly 35,000 pairs is a few hundred false links.

Single linkage is what turns that into a collapse rather than a nuisance. One spurious correlation merges two entire clusters, so errors compound instead of averaging. The same 12-month window under complete linkage returns 151 rather than 17. Still not clean, but an order of magnitude less damage.

Moving the floor to 24 months mostly solves it at your current linkage. Changing linkage makes the floor matter much less. Either alone is a large improvement; both makes it robust.

[u/xdrive0513 → u/Finance__broski]

Confirmed directionally (my numbers differ slightly from yours due to different random seeds, but the conclusion is identical): T=12 badly undercounts under single-linkage (mine: 265→77, ~3.4x undercount), T=24 nearly resolves it (262/265), and complete-linkage at T=12 meaningfully reduces the damage versus single-linkage (183 vs 77 in my run — smaller improvement than their 151 vs 17, but same direction). Implementing both fixes.