Optimization of trading strategy: is there something wrong

What is a central question of all trading strategies is the optimization. However is there something wrong with the optimization or at least with our perception of optimization of trading strategies.

I think there is something elusive here.

I think that the main problem is that we look at the market as a problem.

Yes, I think so, we think that the market is a problem and if there is a problem we look for a solution. Applying mathematical models for searching a solution is just the right thing to do.

But what if, there is something wrong in our paradigm?

Of course the trading rule with the highest observed performance is likely to perform in the future provided we have sufficient observations and we do properly performance statistics.

However the problem is that the best rules are highly positively biased. Aronson writes in his book the Objective Technical analysis that technical analysis methods "have the potential to be valid knowledge but only if back-tested results are considered in light if randomness (sampling variability) and data-mining bias.

This screen shot is from the book of Aronson Evidence based technical analysis. Here you can see out-of-sample performance deterioration, that means that the rule which performed well in the past did not perform in out-of-sample trading.

Here in the first part is the in-sample data: the data used for data mining or simply put rule back-testing. In this period you see 50 % Return on on investment. However the out of sample is a different thing. The out-of-sample actually means that the data is not used for data mining and is the actual performance.

The reality is more complex as we may assume. There is not only a problem that needs a solution to be found but there is something else too. That explains why the data - mining normally fails in out of sample trading. I will stop here for the moment.