NinjaTrader Connector Bug?

[waldoalvarez]

Please check this. I am getting it sometimes with NinjaTrader Coinbase connector. After some connections/disconnection (normal network connections errors), it pops up. First NinjaTrader falls into an inconsistent connection status that fails after some time and disconnects even if you tell to reconnect. In the end, prints error "Adding the specified count to the semaphore would cause it to exceed its maximum count." in messagebox and terminates.

2023-03-04 09:58:09:141 (My Coinbase) Gui.ControlCenter.OnConnectionStatus.Adding: provider=Provider25 status=Connected priceStatus=Connected
2023-03-04 09:58:09:141 NinjaTrader.Gui.ControlCenter.OnConnectionStatus: foundPriceLoss=False foundTradeLoss=False foundConnectingStatus=False foundConnected=True
2023-03-04 09:58:09:142 NinjaTrader.Gui.ControlCenter.OnConnectionStatus: Status=Connected
2023-03-04 09:58:09:216 (My Coinbase) Coinbase.Adapter.SubscribeMarketData1: instrument='BCHUSD' exception=Unable to connect to the remote server
2023-03-04 09:58:09:216 (My Coinbase) Coinbase.Adapter.OnDisconnect0
2023-03-04 09:58:09:673 *************** unhandled exception trapped ***************
2023-03-04 09:58:09:673 Adding the specified count to the semaphore would cause it to exceed its maximum count.
2023-03-04 09:58:09:681 System.Threading.SemaphoreFullException: Adding the specified count to the semaphore would cause it to exceed its maximum count.
   at System.Threading.SemaphoreSlim.Release(Int32 releaseCount)
   at NinjaTrader.Coinbase.Adapter.<SubscribeMarketData>d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
   at System.Windows.Threading.DispatcherOperation.InvokeImpl()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Windows.Threading.DispatcherOperation.Invoke()
   at System.Windows.Threading.Dispatcher.ProcessQueue()
   at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
   at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
   at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
   at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
   at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
   at NinjaTrader.Cbi.Connection.<>c__DisplayClass16_2.<Connect>b__5()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2023-03-04 09:58:14:561 (My Coinbase) Coinbase.Adapter.OnDisconnect1​

[bltdavid]

Which version of NinjaTrader are you running?

[waldoalvarez]

Hi: I am running 8.0.27.1 64-bit but this is not new, looks was present in some previous versions as well. Happens a lot when resuming from hibernation and NinjaTrader was running and connected.

[NinjaTrader_BrandonH]

Hello waldoalvarez,

Thanks for your post.

Please create a clean environment using the direction below and then test to see if you are able to reproduce the behavior in the clean environment. This will allow us to see if it is a platform-specific issue.

Creating a clean environment can be done by following the steps below:

1. Close NinjaTrader 8, and rename the "NinjaTrader 8" folder in My Documents to something like: "NinjaTrader 8 OLD"
2. Uninstall NinjaTrader from the Windows Control Panel and reinstall using the installer from the Client Dashboard > Download page
   1. account.ninjatrader.com
3. Test to see if the behavior occurs in a clean environment.

If you ever need to switch back to your original platform, you may do so by closing NinjaTrader and swapping the platform folder names.

For example, Close NinjaTrader and rename the new "NinjaTrader 8" folder to "NinjaTrader 8 NEW" and the "NinjaTrader 8 OLD" folder to "NinjaTrader 8." Then restart the platform. Simply put: NinjaTrader 8 will always load the "NinjaTrader 8" folder in My Documents.

Are you able to reproduce the behavior in a clean environment? If so, what are the exact steps and settings you used to reproduce the behavior you are reporting so I may look into this further.

Thanks in advance; I look forward to assisting further.

[waldoalvarez]

Just adding some more information, I have tick replay enabled. Can help determine the problem. I develop indicators and usually perform tests but is happening with different ones. Different computers as well. I see in the logs many disconnections/reconnections before the crash.

[kevinop123]

Yes I ran into the same problem

[NinjaTrader_BrandonH]

Hello waldoalvarez and kevinop123,

Thanks for your notes.

Please create a clean environment by following the instruction in post # 3.

Are you able to reproduce the behavior in a clean environment?

If so, what are the exact steps and settings you used to reproduce the behavior you are reporting? I will try to reproduce the behavior on my end using the exact steps and settings you provide and look into the matter further.

I look forward to further assisting.

[waldoalvarez]

I determined the cause, and was caused by a certificate pinning I was doing wrong. It turns out .net intercepts all connections and then all TLS was being intercepted including the ones not related to my server. Looks to me quite bad design on the .net side but that is how things are done. Of course data connections done afterwards were failing certificate validation due to this. NinjaTrader should not miss a semaphore count in those cases however. Looks like an accident waiting to happen anytime. I recommend to review that connector code on your side. This was exacerbated by my mistake but still could happen anytime with normal network connections (failed TLS validation). Seems there is some code path in that connector that leaves a wrong semaphore count in some cases of errors. Regards.