Charts and Analyzer unreal disconnection issue from Database

[NinjaTrader_Brandon]

Hello koganam,
Thanks for your reponse.
Since IB does not provide historical tick data so this should not affect this case. It is not possible for IB to reload the historical data. It should only be accessing the data as it was recorded real-time. It is not expected that this data would change.

[pstrusi]

Koganan, thanks for your contribution. I'm sure that what you're telling is true for that case, but for this it can't explain why all data lived and recorded, that actually looks good and confirmed by backtest, changes itself wrongly after a close of NT platform and restart it again. I mean: while you don't close NT all is Ok, as soon as you close the platform, and if you've done some live simulation previously, then there is risk for this issue.

[koganam]

Hello koganam,
Thanks for your reponse.
Since IB does not provide historical tick data so this should not affect this case. It is not possible for IB to reload the historical data. It should only be accessing the data as it was recorded real-time. It is not expected that this data would change.

Er, not quite. The historical data ends up as the backfill data from the 5-sec extra data that is sent in an attempt to IB to have what they think is a more realistic data stream.

ref: http://ninjatrader.com/support/forum...ead.php?t=2829

[NinjaTrader_Brandon]

Hello pstrusi,
I have not been able to replicate this.
I suspect something on your system is the cause.
I would recommend backing up NinjaTrader, uninstalling, and testing on a new NinjaTrader that has no custom items.
For backup information I have provided a link to the Backup and Restore section of the help guide below.

http://ninjatrader.com/support/helpGuides/nt7/?backup__restore.htm

You can then uninstall NinjaTrader from Windows Control Panel and delete the NinjaTrader 7 folder from your Documents folder.
You can then do a fresh install to test. Ensure not to add any custom scripts for the test.

[pstrusi]

Hello pstrusi,
I have not been able to replicate this.
I suspect something on your system is the cause.
I would recommend backing up NinjaTrader, uninstalling, and testing on a new NinjaTrader that has no custom items.
For backup information I have provided a link to the Backup and Restore section of the help guide below.

http://ninjatrader.com/support/helpGuides/nt7/?backup__restore.htm

You can then uninstall NinjaTrader from Windows Control Panel and delete the NinjaTrader 7 folder from your Documents folder.
You can then do a fresh install to test. Ensure not to add any custom scripts for the test.

Hi Brandon, thanks for your response.

What you've suggested it has been done by me several times in the past, since this issue has been for a quite some time. As I've told before, everything goes fine with the new and fresh installation until a new live simulation is made under the "technical environment conditions previously spoken" ( windows 10-64b, NT7, IB....etc )

I really hope that you and your team don't forget about this issue, and I really hope that this can be prevented in NT8.

Thanks again

[pstrusi]

Hi Brandon and all NT support team.

Last week, despite I didn't perform any live simulation, the issue happened again, but this time I have a very likely culprit. I think what is causing the issue is that during the process of backtests, it has several shown info tabs down, so I noticed that if you click the "Graphs" tab for example to display it, it will take a while before displaying the info. I've begun to think that time where NT looks hanged for a 1 or 2 minutes, it is causing some buffer problem with the incoming live data. This process is the only thing that I did several times last week, and I'd like you to look further deeply on this suspicious performing.

Before closing NT platform ( because I'm always suspicious that the issue will arise eventually ) I did a backup of the data, with the intention to load it if the issue was found. The issue came in, I did the restore but the issue remained. So this indicates me that at that time of that backup, the database was already corrupted by the probably live data buffer problem.

I hope you consider the importance of this matter and trying to help me more proactively, it's in your best interest too.

Thanks in advance, looking forward for your findings

P.S: I've did a backtest, tapping "Strategy Analyzer' s Graph Tab", while downloading live data from IB....so, I closed NT and restart it again....guess what? The issue pops out. So it should be very interesting that you did the same to contrast. I'm increasingly sure that this is it.

[pstrusi]

Identifying and summarizing the issue in this post:

After performing any optimization or backtest while downloading live data in a working day, there's a problem: when you want to look better at performance's results, for example opening the Graph tab down, NT7 will take a while ( around 1 minute ) to display that info, but meanwhile NT7 looks hanged. This behavior seems to affect the incoming live data from your broker or service provider, because from that moment it begins to show a weird behavior, for example tickdata registered in the wrong timestamp, or tickdata simply erased or disappeared.

Another important note: you won't realize the issue until you close NT7 and restart it again, from that point , the distorted database will give unreliable backtest and weird charts, as I've posted earlier with attached jpg files.

So, I'd like that NT support could contrast what I'm telling here, and finding a solution, it's really a frustrating issue.

Thanks in advance

[NinjaTrader_Brandon]

Hello pstrusi,
Thanks for your reply.
Please record a video showing the full steps you are doing as there may be something further missing here.
You can use a free program such as Jing to do this: https://www.techsmith.com/jing.html
You can then upload the video and post the link here.

[pstrusi]

Hello pstrusi,
Thanks for your reply.
Please record a video showing the full steps you are doing as there may be something further missing here.
You can use a free program such as Jing to do this: https://www.techsmith.com/jing.html
You can then upload the video and post the link here.

Hi Brandon.

As you might have read in the last update, I've described the probably culprit, and it's pretty simple for you to contrast it. Just follow the description. it's pretty easy and I don't see any need to do anything to show it. I think that I'm doing quite enough searching on this when I'm just a customer and not a computer professional.

Looking forward for your findings

[NinjaTrader_Brandon]

Hello pstrusi,
Thanks for your reply.
I have followed your steps and I do not see this occur. A video may capture what is being missed from your system to mine. If you do not want to complete these steps we will continue to keep an eye out for cases of this and find the cause.

[pstrusi]

Well, it looks like this issue will never be seen by anyone and even less addressed by NT, pretty frustrating. I've just lost my time writing about it, so I have to deal with it by myself.

[pstrusi]

NT: the best platform but it has a critical flaw

If you go in different traders forums, you probably could check that under my name, I honestly always recommend to juniors traders that using NT is a the most convenient and practical platform for automated trading, because is simple, powerful and reliable enough. Having said that and looking for a better NT platform, I must once again, expose a critical failure that has done harm to my database, that is built from IB tick data, which it means that is unrecoverable for practical terms because this data is different from any other data provider and IB has not historical data service.
Here the issue:

If you're in a normal working trading day, downloading live data, but at the same time you want to do an optimization, this procedure will run perfectly, once the optimization is ready, let's say you want to look deeply into any combination of that analysis, and then you go to some Strategy Analyzer's down-Tab: Graphs, Trades, Periods....etc; once you tap on any of those tabs NT hangs totally and freezes for a while, afterwards then it comes back and you can watch the desired tab info, HOWEVER during this FROZEN time, the incoming live data is registered wrongly in the database, like it was a problem with some incoming data's queue was distorted or altered. This terrible issue is better seen in the 1-tick chart, when you'll see radical, unreal, non-sense and inexplicable price variation, which makes unreliable any future backtest with that data.

I've posted this matter to the exhaustion and still this issue has not had the due attention.

So, if you want to do the test, do this:

- Open NT7 under Windows10 64b
- On working day connect to your realtime data provider, make sure that you don't have any other backup connection to rebuild the database
- Once connected and downloading data, do some optimization.
- When optimization is done, then Tap on any strategy's tab to look further into results
- WATCH how NT do your requirement but it FREEZES for a while
- When NT comes back all seems perfectly well but the incoming live data has been registered wrongly, to check this close NT and restart it again, do the same optimization and you'll see the face of this issue.

I really hope you pay the due attention to this.

Thanks in advance

[pstrusi]

I've finally checked that the root of this issue is whenever you're downloading live data, but you request any procedure that freezes temporarily NT, then something wrong happens to the recording data in the base. Reading old time post in this forum, I've realized that there were a couple telling similar problems. I'm pretty sure that you guys are aware of this problem or limitation. I really hope that this can be addressed in NT8. It's a terrible drawback for Ninjatrader good name. Personally my database is built from IB tick data, which it doesn't have historical backup in case of problems, so I've lost critical info of my system for backtest.

I won't post any additional commentary to this post, the responsibility is on you.

[NinjaTrader_Brandon]

Hello pstrusi,
I will test once again with the additional information you have provided.

[pstrusi]

Thanks Brandon, I really hope that you could reproduce the problem.
One important thing to have in mind: always work with 1-tick chart for better viewing of the issue. If you do the trials with another chart type , you probably won't see any weird fact.

Regards