The Multiple Paths Problem
In real life, you only see one of these lines. You might think you found a "winner" just because your specific path dipped below p = 0.05.
However, looking at 100 parallel simulations reveals the truth: purely by chance, many paths will cross the significance threshold eventually, even though there is no real effect.
The Solution: Sequential Testing
This "peeking" issue is a well-known statistical challenge called Sequential Testing. If you check your results repeatedly, standard tests (like the t-test) become invalid because they don't account for the increased chance of false positives.
Fortunately, this can be addressed using specific statistical methods designed for continuous monitoring.