Data price disruption that makes no sense

[pstrusi]

During the nightly maintenance and the force disconnect you would see a gap in data, no?
To once again confirm, you are using TWS and NOT the IB Gateway? Meaning you have to start TWS in order to connect, correct?

- Night maintenance in IB is around 12:00 am - 1:00am ONLY
- I use the TWS as NT needs

I don't know how much you know about IB but let me inform you, when an auto log-off is triggered the whole connection is closed, it includes the very same TWS ( IB platform ). That is totally different from a brief server reset that IB does every night, that reset time takes around 1 minute, so it's hard to see any gap.

If you don't know much IB, perhaps you could ask for some help among NT support.

I've described a pretty clear situation, easily understood by you guys, c'mon!

[NinjaTrader_PatrickH]

pstrusi,

Unfortunately, this is not a clear cut case. We have exhausted all the items that cause this. And have gone over these items several times. As mentioned before, I would suggest a remote support session but you have stressed that you would prefer not to at this point.

I would leave it up to you at this point. What steps would you like to take at this point? Would you like to run through the usual items again? Would you like to set up a phone call with us here at Platform Support?

[pstrusi]

Hi Patrick,

Thanks for your answer and support with this issue. At this moment, I'll try NOT to reset the Sim101 and also watching if the change in the IB connection ( For historical data : Disabled ) might address it. I'll let you know in this thread if solved.

Thanks once again

[pstrusi]

Hi Patrick,

Now that markets are closed, I closed NT and restart it again, then I've checked my weekly data and it's fine, not issues. One big difference in my procedures was that I have not done any backup as I used to do every week. Also I'm suspicious about doing it directly to a pendrive and not to the pc's hd, so I think that there's a high possibility that this procedure is the causing root of my problem, perhaps during the compressing of the tick database. So, just to discard another possible cause: Do you think that maybe might be some problems doing a backup to a pendrive under Windows 10 and others conditions?

Thanks for your patience regarding to this difficult issue.

[NinjaTrader_PatrickH]

It is possible for data corruption to occur when transferring data to an external drive.

[pstrusi]

I can assure you that my data is not corrupt, simply because I'm building a new one after a fresh install of NT, and still it happens after doing a backup to a pendrive, so my suggestion would be to review that procedure.

Thanks for your attention, I hope this can be solved; meanwhile I'll backup my data in other way.

[pstrusi]

Hi Patrick,

Bad news: once again this frustrating and annoying issue has happened, under two kinds: either of disappearing chunk of data OR simply a total irregular abrupt anormal change of price within the same time, as you can check in the chart attached in this very same post ( for a better visualization I've set the chart to 1 tick chart ).

Good news: This just happens on days when I was doing live simulations with the account sim101, where I had tested performance and reset it a couple of time, a procedure that should not affect my database, but it's happening. I can confirm that the days that I don't run any live simulation, the data is ok.

I can't tell you the havoc that this is causing in my work, mainly because I must keep the integrity of the database with info from the broker ONLY ( Kinetick and so others are not valid for my requirements ), and since I work with IB, it's a big problem cause they don't provide historical data.

Please consider the urgency of this matter, and communicate this issue and the main suspicious reasons to find out what's happening, why and how to solve it.

Thanks in advance

[NinjaTrader_PatrickH]

I will review with development to see if there is anything else we can look for here without a remote session.

[pstrusi]

I will review with development to see if there is anything else we can look for here without a remote session.

By the way, another info is that when I do live simulations, generally I run 4 strategies simultaneously, one for every of the 4 charts for the very same instrument. I thought now, that this maybe could be another reason, check it out, just in case. I'll try to run strategies from the Strategy tab instead, perhaps this could help to discard or find the reason. I'll keep you informed.

Thanks for your diligence and help

[pstrusi]

Hi Patrick,

Few moment ago the issue happened, a brief description:

I close NT totally by accident, and when it was restarted almost in the same minute, the loading chart for EURUSD resumed BUT I noticed that strangely the timestamp in the bottom of that chart was fixed at 10:14:40 ( 4 minutes ahead of actual time ) , and despite it continued to load realtime data, the timestamp was the same, 10:14:40. So when the pc's clock arrived to 10:14:40, the the timestamp in the bottom became normal, updating the actuall time.

How can I explain this?

Thanks in advance

[NinjaTrader_PatrickH]

Hello pstrusi,

Thank you for your patience.

I checked with development and would like to request a few items for our analysis if you do not mind.

First, can you confirm under Tools > Options > Data > that 'Get data from server' is enabled?
Secondly, can you send us your latest Log and Trace files via Help > Mail To Support? Make sure to list 'ATTN: Patrick H - 1424324' in the subject line.
Could you also include the historical data in the e-mail you send out so we may investigate the missing data?

[pstrusi]

Hello pstrusi,

Thank you for your patience.

I checked with development and would like to request a few items for our analysis if you do not mind.

First, can you confirm under Tools > Options > Data > that 'Get data from server' is enabled?
Secondly, can you send us your latest Log and Trace files via Help > Mail To Support? Make sure to list 'ATTN: Patrick H - 1424324' in the subject line.
Could you also include the historical data in the e-mail you send out so we may investigate the missing data?

Yes, Get data from server is enabled

In few minutes, I'll send you out what you request

[pstrusi]

The email was sent, can you confirmed that you've received it with all the requested info ? It took a while to zip it.

Thanks

[NinjaTrader_PatrickH]

pstrusi,

I have the e-mail, thank you for sending those over.

[NinjaTrader_PatrickH]

pstrusi,

I spoke with development and it is possible that with the options 'Save chart data as historical' and 'Get data from server' enabled that we can see the data request remove the prior data when it it s requested and then never replace it as it cannot acquire the data.

Please disable the 'Get data from server option' and advise if the behavior continues with just 'Save chart data as historical'.