Backtesting and live (paper) trades are different

[NinjaTrader_Jason]

@ mart331, could you perhaps start a new thread as per these inquiries. Unfortunately I am not a programmer and very knowledgable with NinjaScript - I cannot comment on this. It would more clear for the thread itself as well, since multiple users post in here.

@ dave1992, yes unfortunately there is no conversion. I will forward your feedback to my superiors.

[dave1992]

@ mart331, unfortunately the Strategy Analyzer can only display currency values in US dollar. For example, the FDAX is set to Euro in the Instrument Manager. However when performing a backtest, the profit or loss in Euro will be reflected in US dollar. Please note that this is an absolute conversion - 1 Euro = 1 US dollar.

There's not even a conversion is there? Backtest doesn't know anything about currencies and just displays numbers. In fact, most of Ninja doesn't do currencies properly does it? If I have a live trade in FGBL and a live trade in YM a 20 euro profit will be displayed as 20 dollars and would be added to the dollar trade to give a completely incorrect total in dollars.

In fact, proper multi currency support rather than a slightly implemented approach has been a suggestion for well over two years. See this thread

http://www.ninjatrader.com/support/forum/showthread.php?t=12198

[mart331]

@ RedEye, unfortunately there are no intra bar calculations when performing backtests in the Strategy Analyzer. Only the OHLC values are used.

@ mart331, unfortunately the Strategy Analyzer can only display currency values in US dollar. For example, the FDAX is set to Euro in the Instrument Manager. However when performing a backtest, the profit or loss in Euro will be reflected in US dollar. Please note that this is an absolute conversion - 1 Euro = 1 US dollar.

Please also reply to my earlier post. Thanks

[NinjaTrader_Jason]

@ RedEye, unfortunately there are no intra bar calculations when performing backtests in the Strategy Analyzer. Only the OHLC values are used.

@ mart331, unfortunately the Strategy Analyzer can only display currency values in US dollar. For example, the FDAX is set to Euro in the Instrument Manager. However when performing a backtest, the profit or loss in Euro will be reflected in US dollar. Please note that this is an absolute conversion - 1 Euro = 1 US dollar.

[mart331]

by the way, the results show profit in $, that should be euro's.

[RedEye]

Hello mart331,

When performing a backtest the indicator values are only known at the close of a bar. While running a strategy in real-time, indicator values are calculated intra bar (Calculate on bar close = False).

Could this be the cause of the discrepancy?

If I'm using tick data and the price is based on the Bid, is there anyway I can have the indicator values calculated interbar?

[mart331]

Unfortunately, I work this way. But I can prove it to you something is wrong.

As you can see in the image,

I add 2 bartypes;
I enter long/short on barsarray == 1 (based on values of barsarray ==2)
So, the tradelogic does NOT have anything to do with the main chart.

In optimize, I ONLY optimize on dataserie, in this case from 1 to 5 minutes,
and you see the differences.... (period = 6 weeks)
from 700 to 10.000 profit

In my view all must be the same.
None of these results equal by far the replay results.

[NinjaTrader_RyanM1]

mart331,

The best example we have for this is this reference sample:
Backtesting NinjaScript Strategies with an intrabar granularity


There will always be differences between a backtest and running a strategy live.

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

[mart331]

As far as I can see:

Backtest:
If your master chart is a 5 minute Kagi chart, but you trade (in strategy) on a 10 second chart (by Add ( second,10), you get enter/exit values from Kagi chart, NOT the 10 second chart values, IS THIS THE CASE? (it should NOT)

I tried to workaround this, by adding 5 minute Kagi chart (AddKagi()) and 10 second master chart, but this also gives very different values compared to replay, a loss instead of 5000 euro profit in replay (6 weeks).

The problem can be the indicator. Because my entry/exit depends on 5 min Kagi, the indicator must use the Kagi data.
Now i use:
if (BarsInProgress == 1)
{
if (Indicator(...) == 1)
enterTrade()
}

I tried (dont know anymore in backtest or Replay) to use Indicator(barsArray[1]), but that gave the same results as in above example.

Tell me if I am doing wrong!

[mart331]

As far as I can see:

Backtest:
If your master chart is a 5 minute Kagi chart, but you trade (in strategy) on a 10 second chart (by Add ( second,10), you get enter/exit values from Kagi chart, NOT the 10 second chart values, and that are barCLOSE or barOPEN values, IS THIS THE CASE? (it should NOT)

[mart331]

no, i use calculateonbarclose=true.
i think and test further.... For now i think all my backtesting and optimation efforts were useless

[NinjaTrader_Jason]

Hello mart331,

When performing a backtest the indicator values are only known at the close of a bar. While running a strategy in real-time, indicator values are calculated intra bar (Calculate on bar close = False).

Could this be the cause of the discrepancy?

[mart331]

Hello, I have another thought on this issue, having the same problem with majoor differences.
I do now test my strategies in replay
Can it be that the indicator values differ a lot? is their data filled in advance with replay and/or backtest?

[NinjaTrader_Jason]

@Camdo, I will forward your suggestion to prevent data being overwritten and only fill missing bars to Development to put it on the list of future considerations of the software. Thank you for your feedback.

@dave1992, there can be minor discrepancies between real-time data and historical data with Zen-Fire in NinjaTrader 6.5.

Zen-Fire does not provide historical data. As a courtesy, we allow Zen-Fire users to make use of our historical data server. Please see the following forum thread.

http://www.ninjatrader-support2.com/vb/showthread.php?p=114191#post114191

As per post 7, data is aggregated differently as per the historical data server. The historical server is based off of NT 6.0 which agreggates tick data into minute bars based on 00:00:01 through 00:00:00. We changed this in NT 6.5 client so minute bars are created from ticks using 00:00:00 through 00:00:59. Thus, if you re-load data you can get subtle differences since there are two difference minute bar compression schemes used.

NinjaTrader 7 is not affected, since its historical data server uses the same compression scheme as the client.

[dave1992]

Couple of points - if data is being changed, presumably it's with data that's more accurate? Secondly, you can avoid any reload by not being connected to your provider when you run the backtest.