How we test a strategy

Most strategy ideas fail. Of every 100 ideas we look at, at most one makes it to live trading. Here is the path a new idea must survive.

Trade-for-Trade Replication. We rebuild the strategy and check that it matches the original trade for trade. If real fills erase the profit, it’s out.
Data Sufficiency Check. We need at least five years of data, including hard years like 2020 and 2022, plus at least 300 trades. Too few trades can be luck.
Out-of-Sample Holdout. We lock away the newest data. No test may look at it until the very end.
Noise Testing. We shake the prices a little, hundreds of times. A real edge still makes money.
Parameter Sensitivity Testing. We nudge each setting up and down. Good strategies sit on a wide, flat area. A strategy that only works at one exact value is fragile.
Walk-Forward Analysis. We tune on one year, then trade the next three months, again and again. Most of those windows must make money.
Random Entry Test. The strategy must beat almost all random-entry copies of itself.
Monte Carlo Permutation Test. We scramble the market’s own price bars and test again. If fake markets do as well, the result was luck.
Transaction Cost Stress Test. We add extra cost to every trade. It must still make money.
One-Shot Out-of-Sample Test. We open the locked data once. Pass, or the idea is done. There is no second try.
Monte Carlo Simulation. Ideas that survive get sized for their worst likely years, not their best.

What this means for you

Testing lowers the chance that a strategy only looked good by luck. It cannot remove that chance.
Even good strategies have losing months.
We keep watching live results, and we retire strategies that stop meeting our standards.