Unexpected behaviour when starting FOREX strategies

[guillembm]

Hello,

I am experiencing some trouble when starting FOREX strategies. Some strategies that require a high value in the "Days to load" setting cannot be loaded sometimes.

This behaviour occurs in any strategy.

The steps for this behaviour to happen are the next:

1. After starting NT8 I enable the strategy.
2. If the strategy doesn't start, I change the "Days to load" setting increasing or decreasing it (it doesn't matter).
3. The strategy starts as expected.
4. I save the workspace.
5. I restart NT8 again.
6. The strategy has the same behaviour.

I also wrote a piece of code at the beginning of OnBarUpdate to see if the data is being loaded like this:

if (CheckStart == false)
{
Print(Instrument.FullName + " Start loading ");
CheckStart = true;
}

When the strategy doesn't start nothing is printed. So, I understand that the strategy is not getting any data. Even the historical data located in the computer.

This happens only with FOREX instruments. Not with securities. It neither happens in NT7.

[NinjaTrader_ChelseaB]

Hello guillembm,

Are there errors appearing on the Log tab of the Control Center?
If so, what is the full error message?

Does the SampleMACrossover included with NinjaTrader experience the same behavior?

[guillembm]

Hello,

I tried several times because it doesn't always happen with the same instruments. I managed to reproduce the error with SampleSMACrossover once using NZDJPY instrument. No errors appear in the log tab. But I noticed two points:

1. It happens more often with Yen-related pairs like NZDJPY, AUDJPY, GBPJPY.
2. If I leave the still not-started strategy waiting to be enabled it loads data directly from the broker (IB) untill, after several hours, the strategy is finally enabled. This behaviour is really a problem for me because when downloading historical data from the servers is gets the data with the volume column empty. Fortunately I saved all the data to recover it after the loss.

No problem with securities using the same strategies.

[NinjaTrader_ChelseaB]

Hello guillembm,

Thank you for confirming this can be reproduced with the SampleMACrossover which would indicate this is not an issue with the script code.

Who are you connecting to for data?
Is all of the historical data within the date range downloaded?
Can you open a chart to the same bar type and interval?

[guillembm]

My broker is Interactive Brokers.

In my computer I have downloaded data since January 2014. I opened a chart for NZDJPY with 2000 days and 400 minutes bar and I can see a chart since 2016. Considering that a year has around 260 working days it seems that I can see all data from the local PC:

[NinjaTrader_ChelseaB]

Hello guillembm,

If there are any gaps NinjaTrader will try and re-download all of the historical data.

Interactive Brokers specifically has a historical data limit.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/platform-technical-support/21731-workaround-for-ib-historical-data-limit?p=270312#post270312

https://ninjatrader.com/support/forum/forum/ninjatrader-7/platform-technical-support/51859-ib-historical-data-request-limit-error

Can you reproduce the same behavior on a chart with 5 Days to load?

[guillembm]

IB has supressed this data limit. The links to the posts you include are old. I know that, previously, I had to load the data by pieces. Not now.

To confirm it I loaded 2 years of data and I had no interruption. I have been loading IB data and I can load as much as I want.

If the gaps you mention exist, how big they must be to force to download the data again? How can I check it?

What I still don't understand is why the strategies with the same instruments sometimes start with no problems and others data is forced to be downloaded. Why there is no problem with stocks or with NT7.

[NinjaTrader_ChelseaB]

Hello guillembm,

From my understanding this limit is still imposed.

I could imagine the historical data cached might be different.

I was not able to confirm, can you reproduce the behavior with 5 days of historical data? (This would be well below the limit)

[guillembm]

Yes,

I tried several times starting strategies with 5 days loaded and this issue doesn't happen. Strategies are starded correctly in a blink.

But I have been trying to increase the value of "Days to Load" and starting with 300 Days to load this issue occurs again. Not always, but it happens. I work with 26 currency pairs and 3 different strategies for each one. It's really annoying that always some of them cannot be enabled.

[NinjaTrader_Jim]

Hello geuillembm,

Interactive Brokers still has strict pacing rules:

https://interactivebrokers.github.io...mitations.html (publicly available link)

The rules may be harder to think of in NinjaTrader terms of how NinjaTrader requests data, but the "do not request too much data too quickly" advice still applies.

When you request data NinjaTrader will look back to when their was last data, so if you already have a good portion of the data, it may not take as long to load.

I do not think you would encounter such issues with your setup if you use a dedicated market data provider service. It may be worth trying if the limits with Interactive Brokers data are burdensome.

[guillembm]

I never request data with less than 1 min. size.

I can assure that I downloaded 2 years of data without interruption. Maybe it's because I did it during the weekend and then IB restrictions are more loose? Or because NT waits patiently for the new set of data after the IB interruption?

There are 3 points I still do not understand:

1. Why this never happens in NT7? Why this never happens with stocks? taking into account that the strategy is the same it should work the same.
2. Why after restarting NT some of the strategies that could be enabled cannot be enabled again or the opposite?
3. Why it tries to load the data anew if the data is complete since 2014? I checked the data and I do not see any gap.

To make sure there are no gaps in data I sent you by mail some data sample.

[NinjaTrader_Jim]

Hello guillembm,

I have given some follow up questions over email.

Please continue corresponding with me there for the questions you have presented here.