Market Replay vs. Back-Tester

[reynoldsn]

I have always used Market Replay and have never tried using the Back Tester but I would like to try it out. Is the same data I download for Market Replay the same data that gets used by the Back Tester?

[NinjaTrader_DrewO]

Hello

Thanks for your post.

Backtesting would actually use Historical data, as opposed to Market Replay data. While similar, these two types of data are fundamentally different in that they are formatted differently. The amount of historical data required to backtest may already be present on your local PC. You can view your historical data in the Historical Data Manager. More information can be found below:

http://ninjatrader.com/support/helpGuides/nt7/historical_data_manager.htm

[reynoldsn]

How does the Historical data got on my machine? Is it copied there when I do a Market Replay download?

[NinjaTrader_DrewO]

Hello,

Historical data, which is fundamentally different than Market Replay data, is typically requested from your connected data provider within the platform, but can also be manually imported.

Market Replay data must be recorded from live data or downloaded using the Download Replay Data utility. This is found from the Control Center under File -> Utilities -> Download Replay Data.

[reynoldsn]

That's what I'm asking - I have neither request the historical data nor manually imported it, and yet when I open Historical Data Manager, I see a lot of CL data which is the instrument I mainly use. How does all this historical data get onto my computer?

[sledge]

He said: "Historical data is typically requested from your connected data provider within the platform, but can also be manually imported. "

You might be able to turn it off in your Account Connection options.

[NinjaTrader_AlexC]

Hi reynoldsn, thanks for your reply.

Historical data is automatically requested and saved to your PC when anything within NinjaTrader requests historical data, such as a chart with a number of 'Days to load' set. Backtesting is another process that requires historical data, and thus the needed amount of historical data will be requested to complete the backtest when the backtest is initialized.

sledge is definitely correct, though, you can disable the downloading of historical data via the Control Center > Tools > Account Connections menu; just select your connection, left-click 'Change', and the option to disable the downloading of historical data will appear on the second Connection Wizard screen, if available. It would also be recommended to navigate to the Control Center > Tools > Options > Data tab and uncheck 'Get data from server (if available)'. Note, your connection won't download historical data any more, so your charts will not populate with historical data nor will backtesting work properly.

[DaFish]

Hi, I'd like to know WHY there are different types of data. Why isn't there just 1 chunk of data (I think there is... ). Historical data is data, Market replay should play historical data. I have a current support issue now where I am being told be NT support that I cannot use HISTORICAL data and I can only test my systems using MARKET REPLAY data.

1) Can you explain why we have 2 seemingly different data streams?

2) Can you let us know what we CANNOT do with Historical and CAN do with REPLAY?

3) What is the difference between REPLAY and BACKTEST ? I know replay uses my downloaded and saved data from my real time feed, what does Back test use?

[sledge]

Isn't this all in the documentation?

Replay allows tick by tick testing and playback.

Historical data is stored in minute bars and does not contain each tick and therefore pretty worthless in a Replay situation that attempts to mimic the live market.

Data providers aggregate at the minute level for charting purposes as it cuts down on data and processing power.

Historical minute data can be useful in testing out ideas pretty fast and optimizing them.

I hope this helps.

[NinjaTrader_AlexC]

Hi DaFish, thanks for your reply.

In addition to the valid points sledge mentioned, users may also want to retain a copy of the historical data provided by their connectivity providers, as well as having a copy of the Market Replay data we provide. The fact that they are contained in separate files allows for this granularity.

[reynoldsn]

Replay allows tick by tick testing and playback...

I didn't see in the docs - interesting, thanks.

[reynoldsn]

Historical data is stored in minute bars and does not contain each tick and therefore pretty worthless in a Replay situation that attempts to mimic the live market.

Data providers aggregate at the minute level for charting purposes as it cuts down on data and processing power.

Historical minute data can be useful in testing out ideas pretty fast and optimizing them.

If what "sledge" says is true, than how is it that we can select chart type in BackTest like "Tick" and "Range" if historical data only stores minute bars?

[sledge]

If what "sledge" says is true, than how is it that we can select chart type in BackTest like "Tick" and "Range" if historical data only stores minute bars?

I was in a rush on my phone earlier.

Check out the 1st 2 options here:

http://ninjatrader.com/support/helpGuides/nt7/?historical_data.htm

You will find tick data goes back so far, generally about 6 months depending on what you are looking for. But minute data goes really far back.

[reynoldsn]

I use CQG. I don't see anything there about 6 months. It just says "Historical Tick Data": YES.
You are saying that Historical data uses minute bars but the link you provide:

http://ninjatrader.com/support/helpG...rical_data.htm

seems to imply that the Historical data is provide as tick data. The third column is called "Historical Tick Data".

[sledge]

I don't use CQG. You should make some tick data requests for something from several years ago and see what is there.

TDA has nothing tick related. I only get tick data during live market and I record it for market replay.