TYPES OF DATA

Commodities pricing data is available for individual or continuous contracts. Individual contract data consists of quotations for individual commodities contracts. At any given time, there may be several contracts actively trading. Most speculators trade thefront-month contracts, those that are most liquid and closest to expiration, but are not yet past first notice date. As each contract nears expiration, or passes first notice date, the trader “rolls over” any open position into the next contract. Working with individual contracts, therefore, can add a great deal of complexity to simulations and tests. Not only must trades directly generated by the trading system be dealt with, but the system developer must also correctly handle rollovers and the selection of appropriate contracts.

To make system testing easier and more practical, the continuous contract was invented. A continuous contract consists of appropriate individual contracts strung together, end to end, to form a single, continuous data series. Some data massaging usually takes place when putting together a continuous contract; the purpose is to close the gaps that occur at rollover, when one contract ends and another begins, Simple back-aajustment appears to be the most reasonable and popular gap-closing method (Schwager, 1992). Back-adjustment involves nothing more than the subtraction of constants, chosen to close the gaps, from all contracts in a series other than the most recent. Since the only operation performed on a contract’s prices is the subtraction of a constant, all linear price relationships (e.g., price changes over time, volatility levels, and ranges) are preserved. Account simulations performed using back-adjusted continuous contracts yield results that need correction only for rollover costs. Once corrected for rollover, simulated trades will produce profits and losses identical to those derived from simulations performed using individual contracts. However, if trading decisions depend upon information involving absolute levels, percentages, or ratios of prices, then additional data series (beyond backadjusted continuous contracts) will be required before tests can be conducted.

End-of-day pricing data, whether in the form of individual or continuous contracts, consists of a series of daily quotations. Each quotation, “bar,” or data point typically contains seven fields of information: date, open, high, low, close, volume, and open interest. Volume and open interest are normally unavailable until after the close of the following day; when testing trading methods, use only past values of these two variables or the outcome may be a fabulous, but essentially untradable, system! The open, high, low, and close (sometimes referred to as the settlement price) are available each day shortly after the market closes.

Intraday pricing data consists either of a series of fixed-interval bars or of individual ticks. The data fields for fixed-interval bars are date, time, open, high, low, close, and tick volume. Tick volume differs from the volume reported for endof- day data series: For intraday data, it is the number of ticks that occur in the period making up the bar, regardless of the number of contracts involved in the transactions reflected in those ticks. Only date, time, and price information are reported for individual ticks: volume is not. Intraday tick data is easily converted into data with fixed-interval bars using readily available software. Conversion software is frequently provided by the data vendor at no extra cost to the consumer.

In addition to commodities pricing data, other kinds of data may be of value. For instance, long-term historical data on sunspot activity, obtained from the Royal Observatory of Belgium, is used in the chapter on lunar and solar influences. Temperature and rainfall data have a bearing on agricultural markets. Various economic time series that cover every aspect of the economy, from inflation to housing starts, may improve the odds of trading commodities successfully. Do not forget to examine reports and measures that reflect sentiment, such as the Commitment of Traders (COT) releases, bullish and bearish consensus surveys, and put-call ratios. Nonquantitative forms of sentiment data, such as news headlines, may also be acquired and quantified for use in systematic tests. Nothing should be ignored. Mining unusual data often uncovers interesting and profitable discoveries. It is often the case that the more esoteric or arcane the data, and the

more difficult it is to obtain, the greater its value!