Importing Historical Data and backtesting Strats

[flotschie]

Hi,
today I downloaded your Demo Version of Ninja Trader. After a few attemps I managed to import historical Data successfully. (1 Year of Tickdata from a 600 MB *txt file)

Doing that some questions came up:

- I already have adjusted data of a Future instrument (FESX on the Exchange EUREX) in the text File. Because of that, I thought it would be the best to create an instrument called FESX_ADJ. Was that the correct way to treat adjusted contracts as one instrument for NT? I do not have data from the single contracts.

-Although the import was successfull, I can not backtest Strategies with the imported Data. I go to File-New-Strategy Analyzer, choose the instrument and a premade Strategy, set some parameters and run the Backtest, but no trades where made. The Range was one Year of Tickdata and I used the SampleMACrossOver Strategy so there should be trades.

- I even can not create Charts of my imported Instrument. Wether I choose Tick Bars, Minute Bars oder Daily Bars, the Chart always stays blank.

Any suggestions are welcome :-)

[NinjaTrader_Ben]

Hello,

I already have adjusted data of a Future instrument (FESX on the Exchange EUREX) in the text File. Because of that, I thought it would be the best to create an instrument called FESX_ADJ. Was that the correct way to treat adjusted contracts as one instrument for NT? I do not have data from the single contracts.
>>By "adjusted" do you mean it is a continuous contract? If so, I would suggest this is the best way to do this. Use the ##-## expiry date in the instrument manager.

-Although the import was successfull, I can not backtest Strategies with the imported Data. I go to File-New-Strategy Analyzer, choose the instrument and a premade Strategy, set some parameters and run the Backtest, but no trades where made. The Range was one Year of Tickdata and I used the SampleMACrossOver Strategy so there should be trades.
>>I am not sure why it is not working. Please post your data file and I will test it for you.

- I even can not create Charts of my imported Instrument. Wether I choose Tick Bars, Minute Bars oder Daily Bars, the Chart always stays blank.
>>I'm not sure what the issue is. Please post your data file and I will take a look.

[flotschie]

These are the first lines of my data file called "FESX_ADJ ##-##.txt"


20071213 08:00:09;4408.6;0
20071213 08:00:09;4408.6;0
20071213 08:00:12;4407.61;0
20071213 08:00:12;4407.61;0
20071213 08:00:12;4407.61;0
20071213 08:00:12;4407.61;0
20071213 08:00:12;4407.61;0
20071213 08:00:12;4406.62;0
20071213 08:00:12;4406.62;0
20071213 08:00:12;4406.62;0
20071213 08:00:12;4406.62;0
20071213 08:00:12;4406.62;0
20071213 08:00:12;4406.62;0
20071213 08:00:12;4407.61;0


Logfile shows only orange messages where it says it replaces the 0- Volume with 100.

And now I can see there is another Message in Yellow saying "Failed to call "Add" method for period type 'Tick': Exception of type 'System.OutOfMemoryException' was thrown." I think this happend because I followed the suggestions in this thread here: http://www.ninjatrader-support2.com/...ad.php?t=11419 So maybe the importing and everything worked well until I tried to create a Chart from the imported data?

[flotschie]

Are there some known memory issues regarding backtesting strategies with historical data containing more than 20 million trade events?

Is NT able to handle tick-accurate backtesting with tick-acurate stoploss-tracing using historical data this size without memory problems, without using compressed bar data (and therefore inaccurate backtest results)?

[NinjaTrader_Ben]

Hello,

Are there some known memory issues regarding backtesting strategies with historical data containing more than 20 million trade events?
>>Not that I am aware of.

Is NT able to handle tick-accurate backtesting with tick-acurate stoploss-tracing using historical data this size without memory problems, without using compressed bar data (and therefore inaccurate backtest results)?
>>Our backtesting, by default, is based on the OHLC of the bars. If you want to backtest and trade on inter bar ticks, you will need to visit this link:

http://www.ninjatrader-support2.com/vb/showthread.php?t=6652

[flotschie]

Thank you Ben, now back to topic - did you already tested the sample data or do you know what I am doing wrong?

[NinjaTrader_Jason]

Hello,

Upon importing the text file, did a pop up message appear stating the import was successful?

Please ensure the .txt file you import has the same name as the instrument name in NinjaTrader. In your situation, you would need to have an instrument listed as FESX_ADJ ##-## in the Instrument Manager.

The crash you reported indicates NinjaTrader was requesting too much resources, since an out-of-memory exception was thrown. Please see the link below to improve the performance of NinjaTrader and reduce resource request.

http://www.ninjatrader-support.com/HelpGuideV6/PerformanceTips1.html

[flotschie]

In your situation, you would need to have an instrument listed as FESX_ADJ ##-## in the Instrument Manager.

Hm, the MasterField only can hold letters and digits and underscore.

If you mean it should be in the Instrument LIST, then yes, I put it in the Default List with expiry "##-##". So in the Instrument Default list it says "FESX_ADJ ##-##".

I import the data from a txt file called 'FESX_ADJ ##-##'. The Message Box tells me that the Historical Data import was successfull.

I go to File-New-Chart and try to make a Chart with 15 Daily Bars - no result, empty chart, every other kombination of tick based charts, minute charts etc also does not work.

[NinjaTrader_Jason]

Ensure the Format Data Series window of the chart you create, holds the time range of the historical data you imported.

I believe you import tick data, please create a chart that uses an interval based on tick data. Please se the following link for more information.

http://www.ninjatrader-support.com/HelpGuideV6/HowDoesNinjaTraderBuildChartBars.html

As per the crash, does it still persists upon creating a chart or backtest a strategy?

[flotschie]

Ah, ok I recieve a Chart. But every time I try to change something, f.e. to create 1 hour candles for several weeks, the chart goes blank again.

[NinjaTrader_Jason]

NinjaTrader handles historical in three types:

- Tick data
- Minute data
- Daily data

Please see the link in post #9 for more information.

It is not supported to display imported historical tick data as historical minute data.

[flotschie]

>>Our backtesting, by default, is based on the OHLC of the bars. If you want to backtest and trade on inter bar ticks, you will need to visit this link:
http://www.ninjatrader-support2.com/...ead.php?t=6652

Questions:

1. If I backtest a strategy using 30 Minute Bars, and I use a StopLoss technique to exit a trade, how does it work in NT if I am NOT using a second bar series?

2. In your sample code in Line 53-55 it says

///<summary>
/// Called on each bar update event (incoming tick)
///</summary>
protectedoverridevoid OnBarUpdate()
{

and in the documentation it says:

Base Data used to Build Bars
A chart bar (interval type) requires a base data value as its source for bar construction. Following are NinjaTrader supported interval types and their required base data values.


Tick - tick data
Volume - tick data
Range - tick data
Second - tick data
Minute - minute bar data
Day - daily bar data
Week - daily bar data
Month - daily bar data
Year - daily bar data

So - please explain this: OnBarUpdate is called with every incoming tick. Minute Bars are filled with minute bar data. How can they update with every incoming tick in Backtests?

[NinjaTrader_Bertrand]

Hi flotschie,

1. The StopLoss will work intrabar, please also see this link - http://www.ninjatrader-support.com/H...tStopLoss.html

2. In realtime you can set the OnBarUpdate() behavior with CalculateOnBarClose - http://www.ninjatrader-support.com/H...dateEvent.html

For backtesting I would suggest using this approach - http://www.ninjatrader-support2.com/...ead.php?t=6652