Worst Case Analysis

Somebody once asked me “if there was only one performance report you could look at to make a decision about a system what would it be?” My first reaction was that this was a silly question. There are lots of factors that must be considered when choosing a trading system. There are countless performance indicators and ratios. Things such as average yearly return, maximum drawdown, Sharpe ratio, margin requirements, robustness, the list is very long. However, there has indeed been one report that I have come to rely on more than any other report. It’s a report that has given me more comfort and confidence as a trader than any other report. In fact, if I knew a system was properly created I could almost use this report alone to make a decision about trading it! So what is this report you ask? It’s called a “Start Trade Report”.

In my opinion, the start trade report gives you the most robust three dimensional view of a system possible. It cuts through so much of the pitfalls in traditional analysis. It even cuts through the nonsense involved in looking at real time performance. You may be thinking “wait a minute, how can you argue with real time performance?” Let me give you an example with one of my systems Synergy. In May of 2003 Synergy initiated a trade in London Copper. The trade became the most successful trade of the year. As of this writing, (March 7th 2004) the trade has profits of over $25,000 per contract. If you were using position sizing you might have had on 2 or 3 (or more) of these contracts. However, had you started a week or even a day after this trade you would have missed it! Therefore, two investors trading the exact same system with the exact same amount of money and the exact same money management rules could have a $25,000 or $50,000 or $75,000 (or more) difference in their account! AND they might have only started one day apart. This can create tremendous confusion because one broker’s real time accounts could be far different than another brokers real time accounts both trading the same system.

This phenomenon can also be used for disingenuous purposes. It’s possible for a systems vendor to cherry picked an optimal historical starting date. He can choose a date right before a huge winner (or series of winners). This can make it look like the system needed very little original starting capital and that the return on invested funds was enormous. Basically you financed trading with your initial winners. However, what if you had started on a different date? What if you had started on a date that was right before a series of losers? You might have needed 2 or 3 or 4 times the starting capital than you would have starting on a different date. Therefore, the return on invested capital would have been much less, or, you might have lost all your investment before making the profits shown.

Even if a broker or vendor shows an average of a number of accounts this can still be a very limited view. They could still cherry pick the 3 or 4 accounts and their different starting dates. Or they could have so few accounts to average from that the data suffers from what statisticians call a small sample size. Basically this means not enough data to draw any valid conclusions.

The worst offender would be if a disingenuous brokerage or vendor was pushing a day trading system because of the high frequency of trades and commissions it generated and then used some cherry picked “real time” accounts to “prove” that it worked.

The point I am basically making is that thee are countless ways that start dates can impact performance both in hypothetical reports and also “actual” real time performances.

You need to have something far more robust.

What’s the answer you ask?

Well, in my opinion it is the start trade report. What the start trade report does is tests a system hundreds or thousands of times over the given period. Each test it starts on a new date that coincides with a date that you could have taken a new trade. If there were 2000 trades over a 10 year period then it will retest the system 2000 times starting on the date of each new trade every time. In addition, it resets the equity back to the original starting amount with each test. This is important because when using position sizing you may skip certain trades in the beginning when the equity is small. It’s not correct to look at the results of trades that you would not have taken. I have sometimes seen brokerage firms report on trades my system made that many of my clients would not have taken based on their account size. For example, a $3,500 losing trade in a system where most clients would have skipped any trades with risk above $2,000. The start trade report knows to skip trades at the right time based on your starting amount. This report can also let you evaluate performance based off of the margin required. What this allows you to do is see ALL the outcomes, instead of just one.

A few things a Start Trade Report can show you are:

1. What percentage of the first 12 months were profitable over 2000 different starting dates?
2. What was the average first year performance when averaged over 2000 different starting dates?
3. How much money did I need in my account if I started on the worst possible date?
4. What was the average account size I needed to trade the system over 2000 different starting periods?
5. What was the average and the most I ever went under my original starting amount? (This is different than maximum drawdown)

The list goes on and on. This report allows you to filter out so much of the garbage seen in typical performance reporting. It filters out so many of the errors in reporting “real time” performance based on a limited sample size or “cherry picked” starting sates and accounts.

Hopefully you can see that this report is invaluable. I honestly don’t know how you could ever trade any system without it. You can see how much comfort and confidence this can build when you have looked at a system in this much detail. When I first started trading for myself this is he report that gave me incredible peace of mind. It was the only report that comforted me when there were drawdowns. It allowed me to know whether or not we were in the normal ranges of the bell curve. It also gave me a very realistic range of outcomes to expect within the first year of trading.

We feel that providing you these reports gives you an incredible edge and builds confidence. You need this confidence when the inevitable drawdown comes. In my own personal case Im able to remain very calm during those times because of these reports. To get a copy of our start trade reports please email us.

Source: www.traderstech.net