Chart data in Strategy Analyzer does not match real chart data

[Greg1]

I am having problems with the strategy analyzer. Basically, I run a back test against ER2 09-07 3 minute time frame. Time frame is set from 9:30 to 4:00. I've attached an image called backtest which shows the chart data from strategy analyzer.

I've also attached an image called chart which shows the actual chart data also on a 3 minute timeframe which is correct (from this past Friday).

You can see that the chart data between charts and strategy analyzer are completely different. For example, going into the close general down bias but strategy analyzer chart data has a generally up bias.

Thus, obviously making me question the results from the back tester.

What am I doing wrong ? Note that I have closed NT, restarted it and same problem exists. I am running the latest NT software.

[NinjaTrader_Ray]

The data in the two charts are different for sure suggesting that they are either not the same data set or you are comparing two different days of data.

Open up two charts via File > New of the same range and type and they charts should be identical. Assuming they are, then assuming you use the same settings in the backtest, the charts will be identical as well since it goes to the same data pool.

[Greg1]

Yes, that is what is baffling.

I opened up two new charts on ER2 09-07 with 3 minute timeframes, exclude weekends, 9:30 AM to 4:00 PM.

Then, I went to a new Strategy Analyzer window, right clicked on ER2 09-07, clicked on BackTest and made sure the settings were the same (3 min, exclude weekend, 9:30 to 4:00). Chart that displays in the Strategy Analyzer is just not the same data.

I agree with your analysis. The charts should be identical but they just are not...

[NinjaTrader_Dierk]

Please post 2 screenshots of 2 charts where the series do not match (including the window frame!) and point out the differences. Thanks.

[Greg1]

I tried to attach a Word document that shows the problem but the file is too large to upload (is there an email address that I can send the document to?)

The attachment will show:
1) A screen shot of ER2 377 tick data that shows the nice sharp decline going into Friday's close. Along with the chart properties dialog that shows chart settings
2) A screen shot from strategy analyzer using ER2 377 tick data. Problem is that it shows prices going up into the close which obviously is not correct. I've also attached the backtest dialog that shows the settings used.

Also, note that this problem comes and goes and I have not found a pattern to reproduce. I had reported this problem a few months ago and then it worked for awhile. Then it stopped working and I gave up on back testing stuff in Ninja Trader.

Last week I tried again and had the same problem. Problem disappeared for a couple of days and it's back again today.

Unfortunately, I know how hard it is to fix things that aren't consistently repeatable but it happens frequently enough. At one point I thought I was crazy so I showed my live charts to a few other traders (that also use Ninja Trader) and they thought I was doing things correctly and were surprised at the discrepancies.

[NinjaTrader_Dierk]

We don't have any similar problem report for now. You may consider zipped your word document and attaching it then.

[samlam]

Backtest indicators still have discrepency with charts.

Greetings, I have the exact same issue than Greg1.

Basically I am taking decision over the value of two indicators (BB,SMA). last Friday I runned a backtest in which I print the value of each BB and SMA as the bars occurs (working on daily bar). There was decrepency between what the Backtest used and the information shown on the chart.

Sadly, I have not noted the values I received from the Backtest, but I did write down the ones from the chart. I have the exact same value from last Friday chart and today chart for 24/02/09 (so i guess all the other match as well). So basically, the Backtest values used, corrected themselves somehow.

Also, I do not think it's relevant, but basically, after I noticed that last Friday I gave up for the day and did not touch the code till today. When I got back on it today I added 2 lines of code in the Initialize() method:

Add(Boillinger(BarsArray[0],#,#)); and Add(SMA(BarsArray[0], #));

Obviously, this compiled, but as soon I tried to configure it, it crashed (Oh yea, can't use BarsArray in Initialize(), oups! I need more coffee..)

So I commented those line and redid the backtest to try to analyze the values. To my surprise, the values matched the ones from the chart..

I do not know if this Matter, but I am using IQFeed. Oh and it was not just one discrepency here and there, all of the values were wrong as all of the values are right today.

[NinjaTrader_JoshP]

samlam,

Add() for indicators does not influence anything from a data perspective. What exactly are you comparing? Real time chart versus historical chart? Data providers can have different data sets depending on which server you end up pulling data from. What you need to do is just connect to IQFeed once, open a chart of your instrument to download data, and then disconnect. Now what you need to do is run your backtest and then compare values. You will have 100% the same data set.

Remember, any indicators you add onto a chart are always in relation to the primary bars object or whichever instrument & time frame is actually being charted. If you are printing values from a different bar series for sure the values will not match up.

[samlam]

That is exactly, my though that the Add() method shouldn't not influence it at all. I am also award that the settings should match, it's the first thing I verified.

If I understood what you are saying correctly, the only way to backtest with an historical data that do not change is to do them offline? Also, unless i missed something, this mean that you cannot have strategy running while you are backtesting several cases with a strategy?

[NinjaTrader_JoshP]

samlam,

No, you can be connected and running strategies. This step is to ensure you are absolutely comparing apples to apples. There is nothing you or we can do to control which data server you actually pull data from. All we can guarantee is that provided you have the same data set on the same strategy you will have the same results. By same results, I do not mean real-time would perform the same as backtest. I mean running multiple backtests will produce the same result.

http://www.ninjatrader-support.com/HelpGuideV6/helpguide.html?DiscrepanciesReal-TimeVsBacktest

[samlam]

Exactly, you can do both, although there is a big question mark to it. Since between each backtest NT try to pull Data from the server (at least it verify the conditions if it need to do so), the only way to guarnateed that 2 backtests in a row use the same values is to be disconnected. Otherwise, for example one data server have a gap for date X and another has date X but not date y. Thus, you can end up reloading the data everytime. Is this possible? Another example, you run a serie of backtests to which you decrement the "from" by 1 bar every pass.

[NinjaTrader_JoshP]

samlam,

You always run the possibility of reloading historical data which may or may not change if you are connected and run into any of these scenarios: http://www.ninjatrader-support.com/H...ricalData.html

If you are concerned about this, you have to disconnect to ensure a standard set of data.

[samlam]

thank you for clarifications