backtest results change after re-loading history

[waltFX]

Ok, so I created a strategy on my dev machine and back-tested with a return of 5k...
Then I exported my data and moved my strat to the prod machine and imported the data.
The back-test result on the prod machine was 2k... same code same settings.. only difference of the PC's was slower processor and less memory.

So then I wen back to my dev box and imported the data (exported previously) and now my back-test agree but only 2k.

Where is the disconnect? What should I do to get consistent results??

[NinjaTrader_ChelseaB]

Hello waltFX,

If the code is the same, the data series is the same, and the data itself is the same, the results will be the same.

If there is a difference, one of these things is different. It could be the data series (such as date range, or differences in trading hours, etc), or could be the data is not the same.

To find out what is causing differences, you can write the data to file.

Below is a link to a forum post that demonstrates.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance#post100192

[waltFX]

I think you are missing the issue.

The data from the dev box (5k result) was exported and then the SAME file was imported back into the dev box and the results (2k) were different from the original dev results...

I did not do anything with the file except import the data back into the source box. This should NOT have changed results...

[NinjaTrader_ChelseaB]

Hello waltFX,

Something is different. Find out what is different.

[waltFX]

Something is different. Find out what is different.

Seriously!! That is not an answer...

Please don't answer if you don't have any suggestions...

[NinjaTrader_ChelseaB]

Hello waltFX,

My suggestion is to write the information used in the conditions to file and see what is different. Mostly likely the output in the file is going to show you where the data is different, and may show there is an issue with importing.

Once you have the output, we can move forward.

[waltFX]

• I "printed" O H L C from a script and saved it to a file
• then exported the same data using the NT "export" feature
• these two files SHOULD be the same but were significantly different.

So, the issue is that that data exported does NOT have the resolution required for forex data which is significant to 5 decimal places; .00001 (which is the "TickSize").

Needless to say, this is very disappointing as I did not want to spend my afternoon debugging...

>> I would have attached an Excel with samples but the file type is not allowed...

[NinjaTrader_ChelseaB]

Hello waltFX,

To see what the differences are, you would write to a text file the time of the bar and all values used in the conditions that place orders.

You would run the script and get the text file from the first computer, then run the script and get the text file from the second computer. Then compare the files with diffchecker, as shown in the video in the forum post I provided you.

To confirm, you are following the steps in the video, is this correct?


Regarding exporting historical data, I am not able to reproduce.

I've exported EURUSD minute data and I am finding the exported historical has the correct number of decimal places.

Below is a link to a test video.

https://drive.google.com/file/d/1raVHt6JwZzgbHDdP7krfVnfAs2-t30c6/view?usp=drivesdk

[waltFX]

Look at what I wrote; I "printed" every data point to a file and exported the data to a different file AND THEY DO NOT MATCH!
Conditions have NOTHING TO DO WITH IT!
If the data does not match the results will not match.

[NinjaTrader_ChelseaB]

Hello waltFX,

I am not able to reproduce on my end. I've tested exporting data and importing this and am finding the data is the same.

[waltFX]

Sorry but when you export and import the same data through the same pipeline, of course it will be the same...

I looked a over 219000 data points and they do not match.

Just forget about it... I'm tired of beating a dead horse..

This conversation is over...

BTW; I have been doing databases and high volume data for over 30 years... I think I know just a bit about data...

[NinjaTrader_ChelseaB]

Hello waltFX,

Is your claim that the data from the exported file, is not matching what is in the historical data window after it is imported on a new machine?

Can you provide the exported historical data file and screenshots of where this is not matching in the historical data window, along with your time zone?

If not, let's schedule a call where I can perform the tasks I have requested output from on your behalf.

Please send an email to support [at] ninjatrader.com. In the email please include a link to this forum thread.