Performance Summary Reports

As an illustration of the appearance of performance summary reports, two have been prepared using the same moving-average crossover system employed to illustrate simulator programming. Both the TradeStation (Table 2-l) and C-Trader (Table 2-2) implementations of this system were run using their respective target software applications. In each instance, the length parameter (controls the period of the moving average) was set to 4. Such style factors as the total number of trades, the number of winning trades, the number of losing trades, the percentage of profitable trades, the maximum numbers of consecutive winners and losers, and the average numbers of bars in winners and losers also appear in performance summary reports. Reward, risk, and style are critical aspects of system performance that these reports address.

Although all address the issues of reward, risk and trading style, there are a number of differences between various performance summary reports. Least significant are differences in formatting. Some reports, in an effort to cram as much information as possible into a limited amount of space, round dollar values to the nearest whole integer, scale up certain values by some factor of 10 to avoid the need for decimals, and arrange their output in a tabular, spreadsheet-like format. Other reports use less cryptic descriptors, do not round dollar values or rescale numbers, and format their output to resemble more traditional reports,

Somewhat more significant than differences in formatting are the variations between performance summary reports that result from the definitions and assumptions made in various calculations. For instance, the number of winning trades may differ slightly between reports because of how winners are defined. Some simulators count as a winner any trade in which the P/L (proWloss) figure is greater than or equal to zero, whereas others count as winners only trades for which the P/L is strictly greater than zero. This difference in calculation also affects figures for the average winning trade and for the ratio of the average winnerto the  average loser. Likewise, the average number of bars in a trade may be greater or fewer, depending on how they are counted. Some simulators include the entry bar in all bar counts; others do not. Return-on-account figures may also differ, depending, for instance, on whether or not they are annualized.

Differences in content between performance summary reports may even be more significant. Some only break down their performance analyses into long positions, short positions, and all trades combined. Others break them down into in-sample and out-of-sample trades, as well. The additional breakdown makes it easy to see whether a system optimized on one sample of data (the in-sample set) shows similar behavior on another sample (the out-of-sample data) used for verification; out-of-sample tests are imperative for optimized systems. Other important information, such as the total bar counts, maximum run-up (the converse of drawdown), adverse and favorable excursion numbers, peak equity, lowest equity, annualized return in dollars, trade variability (expressed as a standard deviation), and the annualized risk-to-reward ratio (a variant of the Sharpe Ratio), are present in some reports. The calculation of inferential statistics, such as the t-statistic and its associated probability, either for a single test or corrected for multiple tests or optimizations, is also a desirable feature. Statistical items, such as t-tests and probabilities, are important since they help reveal whether a system’s performance reflects the capture of a valid market inefficiency or is merely due to chanc  or excessive curve-fitting. Many additional, possibly useful statistics can also be cnl culated, some of them on the basis of the information present in performance summaries. Among these statistics (Stendahl, 1999) are net positive outliers, net negative outliers, select net profit (calculated after the removal of outlier trades), loss ratio (greatest loss divided by net profit), run-up-t&rawdown ratio, longest flat period, and buy-and-hold return (useful as a baseline). Finally, some reports also contain a text-based plot of account equity as a function of time.

To the degree that history repeats itself, a clear image of the past seems like an excellent foundation from which to envision a likely future. A good performance summary provides a panoramic view of a trading method’s historical behavior. Figures on return and risk show how well the system traded on test data from the historical period under study. The Sharpe Ratio, or annualized risk to Reward, measures return on a risk- or stability-adjusted scale. T-tests and related statistics may be used to determine whether a system’s performance derives from some real market inefficiency or is an artifact of chance, multiple tests, or inappropriate optimization. Performance due to real market inefficiency may persist for a time, while that due to artifact is unlikely to recur in the future. In short, a good performance summary aids in capturing profitable market phenomena likely to persist; the capture of persistent market inefficiency is, of course, the basis for any sustained success as a trader.

This wraps up the discussion of one kind of report obtainable within most trading simulation environments. Next we consider the other type of output that most simulators provide: the trade-by-trade report.