How to auto manage IB-Gateway daily restart without causing NT8 to stop working

[NinjaTrader_Jim]

Hello timko,

Orange means an error unless you have changed this in the Properties menu.

What is the full error you received here? You can highlight the row and use Ctrl + C to copy the error.

Can you also confirm if you are on Release 24.1? You can check the Control Center's Help > About menu to confirm.

[timko]

Thanks Jim! I didn't change the default Property.
That means, NT is in error while GW still thinks it's a good connection.
FYI, under such a situation, orders cannot be placed,

NT 24.1

[NinjaTrader_Jim]

Hello timko,

Could you copy the full error code seen? This may be a different symptom. When I could reproduce, my connection was red and I did not see any errors.

[timko]

Jim, this is the same symptom I reported (in words) the second day after 8.0.23.0 was released. this happens again in 24.1 and i made a screenshot for you guys because i read a suggestion to use Gw937.2h as a solution. that's misleading and 937.2h is definitely not a solution. the orange light is even more mysterious than red with gw showing green API!!!

[NinjaTrader_Jim]

timko, I was asking about the error's information under the Message column. This is not included with your posts. If it is, could you direct me?

Using IB Gateway 973.2h is something I have heard others report as a work around. The solution we would provide would involve changes to the Interactive Broker Connection Adapter in NinjaTrader and would involve using the same version of TWS/IB Gateway we note in the Interactive Brokers Connection Guide page. Currently that version is 978.2c

We do not have a solution ready for those issues and I am going to post back here when progress is made.

If the error that you mention is using TWS/IB Gateway 978.2c with NinjaTrader 8 Release 24.1, I would like to forward that information to our Quality Assurance and Development teams.

[Coral]

Just a quick Update from my side: I now installed back the old versions: NT 8.0.22 and old IB Gateway 973.2h. In that combination the IB disconnects that are not able to reconnect automatically over night do not occur any more.

But please keep us updated when it's fixed with the new versions of Ninjatrader & IB Gateway...

Should not be a long term solution to fall back on old versions ;-) Thank you to keep up on the case!

[timko]

Jim, after a few weeks' investigation using a spare live real money IB account, I made the following observation:
1. the issue has a over 75% probability to occur on Friday mid night when IB performs a weekend maintenance
2. This maintenance spans from 1 hour to a few hours, generally from around 1am on saturday and ends around 3 to 4 hours later.
3. during the maintenance period, the GW software keeps to reconnect and would 100% perfom a successful reconnection to IB (green light shown in GW software). Please take note that the official IB maintenance lasts more than 3 or 4 hours. I am not sure this successful reconnection was to a live server or a backup server.
4. upon the GW to IB server reconnection after a few hours' retry, NT has a 50% successful reconnection to the GW software. Pls take note, NT was in full live trading mode, ie. with a strategy running under the live IB trading account. 50% successful reconnection means
a. a successful green light reconnection and straetgy runs normal (like after a brief drop of IB GW disconnection due to a brief drop of the GW-to-server link) OR
b. an orange light (strategy seems ok but indeed, when an order is placed, it will not be successful), OR
c. NT retrys forever to reconnect to GW software.

I have the feeling that 4a or 4b happens on and after high volatility days. feeling only.
It's more and more obvious to me NT's reconnection to GW is not solid. GW used is 973,2h (the version you guys suggested as a workaround). 973.2 users please be cautious.

Also, Kinetick reconnection can be fuzzy and does not support FAST reconnections. This is KILLING as in order to stop the reconnection retry, a disconnection to Kinetick has to be made manually and a new connection to the Kinectick datafeed would cause EVERYTHING to be recalculated. orderflow charts, strategy logics etc all gone and have to reset or restart again. It's devastating.... I have suggested to implement a retry delay user setting but you guys simply ignore my suggestion. As Kinetick is under NT's business, that is why I am posting in here, Pls inform me if Kinetick issues have to moved to somehwere else. I don't want to be another Apple Mac version requester.

Sorry that I only share what I observed. No need to ask me for logs as you guys have a full fleet of machines with different configurations and the benefit of being a developer. The logs are very simple as your design team made them to... and basically useless... i believe my observation is much more valuable than your log lines. pls don't think that i am too mean. Think about this core issue has been lingering for nearly a year now. An ***Apple ARM Mac version*** can be finished from ground up within this period of time if I have to roll up my sleeve. Let alone those small modifications (ignoring @ when installing add-ons, bulk import of add-ons, bulk download of historical data etc) that you can be made to make user's life easier, happier.

[timko]

In addition. Ninjatrader Continuum (free) + free Ninjatrader does not have the same issue as my subscribed Kinetick + licensed Ninjatrader. Is this sarcastic?

[pmn100]

Just a quick Update from my side: I now installed back the old versions: NT 8.0.22 and old IB Gateway 973.2h. In that combination the IB disconnects that are not able to reconnect automatically over night do not occur any more.

But please keep us updated when it's fixed with the new versions of Ninjatrader & IB Gateway...

Should not be a long term solution to fall back on old versions ;-) Thank you to keep up on the case!

This is the only work around at the moment. 8.0.22 survives the nightly disconnects but unfortunately does not auto connect when I have to restart TWS every 24 hours. This has been an issue for several months now and very disapointing NT hasn't got to the bottom of it. 8.0.22 survives the daily server reset at IB so surely the solution is there, but if I understand it correctly, the developers won't move on an issue unless it can be specifically repeated by support staff, so we're left in limbo for months with no fix on the horizon for something the devs broke in an update.

[Coral]

Another Update from my side:

This morning even with the old versions: NT 8.0.22 and old IB Gateway 973.2h. after the IB connection reset the system was not able to automatically reconnect any more while last week it has worked.



Can we follow up somewhere (under a specific link or so) how far the progress of the issue stands?

[NT-Roland]

Hi,
This is so disappointing. We are all sitting in the same boat and suffering. I will stick to the principle not to promote any other software in this forum.
However, I consider this a serious bug that I can't leverage my NinjaTrader Multibroker lifetime license as expected. My other platform is rock solid, no issue whatsoever with disconnects (same broker, also via TWS, same datafeed, same internet connection, same installation environment). Thus, it is obviously possible to successfully re-establish the connection to the broker and datafeed no matter what.
NT-Roland

[NinjaTrader_Jim]

Hello everyone,

I can't provide any insight for using the older version, but that sort of discussion is welcome between community members to discuss workarounds.

As for the current version of NinjaTrader 8 and TWS/IB Gateway 978.2c, Development has pushed a fix in our internal build to help address these issues. The developer who applied the fix was not able to reproduce the issue with their tests before working on the issue, but I can share that I have given a few over night tests on my end with this build and I have not seen the reconnect issue occur.

For everyone affected by this issue, please email me at platformsupport [at] ninjatrader [dot] com with the text "Attn Jim - 2665543 New Ticket" and please include a link to this thread. Please also include the following:

1. Log and Trace Files
2. A description of how often this is encountered. (Every night, every restart, noticing during certain hours, etc.)

Log and trace files are located in the Documents\NinjaTrader 8\ folder. Please zip the log and trace folders and attach them to your email when writing in.

I can then provide an installer for our internal build so you may test as well and confirm if the issue is still present.

I look forward to hearing from all of those who are affected.

[timko]

I can still vividly remember I heard the similar saying before, "Development has pushed a fix in our internal build to help address these issues. The developer who applied the fix was not able to reproduce the issue with their tests before working on the issue, but I can share that I have given a few over night tests on my end with this build and I have not seen the reconnect issue occur.".

That day, I rushed myself to do all the updates (after losing some money due to the connection issue) and my frustations came days later. It was a nightmare as downgrading took me even a lot of time and energy to upgrade.

Jim, it's not personal, I understand you are a very nice and helpful person. But could you tell me, had your development team found the *root cause*? Identifying the root cause is the first step. The next step is to fix it without inducing other problems. Forgive me but I hope you understand how frustrated, disappointed it is when the issue has been lingering for a year now...

Please get it fixed! Please.....

[NinjaTrader_Jim]

Hello timko,

Determining the exact cause requires being able to reproduce the issue while taking debugging steps. Unfortunately, this issue does not occur consistently where it can be well studied. We understand it is happening, but since the issue cannot be well studied, it makes it very difficult to resolve.

Your feedback testing the internal build will be helpful for us and for others to put this matter to rest. If we get feedback that the issue is still happening with the internal build, we can let Development know so they can continue working the issue.

I understand your frustration. Right now the best path forward is to work with us and let us know if the issue still occurs with our internal build. I will happily provide it if you email me.(See post #42)

[pmn100]

I received your email Jim and echoing Timko's concerns I tested an internal build from you several weeks ago that suggested a fix but failed and I ended up losing $$$ of real money due to not being in positions I should have been in. The email suggests a full clean install, that takes a lot of time setting up multiple charts, timeframes and applying strategies etc and then testing on a live account with substantial currency at stake. Its a big ask.

Nevertheless I will attempt this latest build and do what I can to make sure I'm awake in the middle of the night to catch the disconnect if it occurs. But this will be the last time for me. As a poster above alluded to, Ninjatrader has competitors that do not have these issues with the same Brokers and I'm gutted at this stage I didn't start out with them because Ninjatrader bugs have literally cost me thousands of dollars over the last 3 years.