A look At Optimization

To the new systems developer one of the most exciting things to play with is optimization. Optimization is using the power of the computer to examine every possible sequence of parameters and rules to find those that have worked out best in the past. With enough computer crunching power its possible to find systems that perfectly “predicted” the past. We can run number crunching PC's on automated routines and have them analyze billions of bits of data while we are sleeping! Many traders do this long enough and eventually "discoverer" the holy grail of trading systems. They jump into the markets with their new super predictive algorithms only to find they fall apart in real trading!

“What happened?” they ask themselves. The answer is that what they created was likely a system that was a statistical coincidence (known as a "curve fit"). Curve fitting is where a system has been optimized to a unique set of historical data. The problem is that the markets will behave much differently in the future than the past, therefore, a “perfect” trading system in the past could be useless in the future. For example, your computer finds the perfect dates in the past to have bought and then sold the market. Obviously this data does not mean anything in the future. . This is a simple example but most curve fits are some complex form of this basic concept.

Lets look at another flawed example. Assume we wanted to optimize a set of nickels that were most likely to land on heads. What we could do is flip a million nickels and only select those that landed on heads. Then, we can take those remaining nickels and flip them again, once again only choosing those that land on heads. We could repeat this process over and over again each time only choosing those nickels that land on heads. At this point we might conclude that we had narrowed down our nickels to only a small handful that were “optimized” to land on heads. We could then go out and make large bets with those nickels putting all our money on heads. We would quickly make a fortune right? WRONG!

Unfortunately we would very quickly lose our money. These nickels were not optimized for heads; they always did and always will have 50/50 odds. What might have confused some is that they thought they had found a predictable set of nickels when in fact they had just found a statistical coincidence!

Because there is so much data and so much computing power available, these kinds of errors find there way into trading systems all of the time. One of the worst offenders of such flawed optimized systems can be neural networks. When developing a system its imperative that optimizing is avoided as much as possible. You need to find NON curve-fit robust systems. There can be a place for certain types of optimizing, but it must be handled correctly.