NT Locking Up

[Shinjitsu noDeshi]

Hello -

I upgraded recently to NT 8.0.27.1, and IBKR TWS 10.19.1k

Since then, NT has been locking up, with the charts and strategy running on it becoming unresponsive to data updates or mouse clicks. It appears to occur randomly, but has happened almost every day since the upgrade.

Attempting disable the strategy from the control center will show the strategy stopped, but the chart remains unresponsive. Eventually, most of NT will simply lock up. If I shut it down and attempt to restart it, I receive this message:



I am not using any multithreading in my strategy, so I'm not programmatically acquiring/releasing locks.

I need to end NT's process from the task manager in order to restart the program.

There doesn't appear to be any errors related to the lockup in the control center logs, or the ninjascript output.

Any help identifying the issue causing this would be appreciated.

Thanks

[QuantKey_Bruce]

This has been reported a lot.

You might want to read (these are just the first few... there are more...)
https://forum.ninjatrader.com/forum/...ite-lock-error
https://forum.ninjatrader.com/forum/...rsion-8-0-27-1
https://forum.ninjatrader.com/forum/...-ninjatrader-8
https://forum.ninjatrader.com/forum/...k-error

[NinjaTrader_ChelseaB]

Hello Shinjitsu noDeshi,

This could be caused by OneDrive, or other cloud syncing services or antivirus software, or could be from a custom coded script.

Please close all open workspaces, then in a new blank workspace open a new 1 minute chart with no template applied and do not apply any indicators.

Test the SampleMACrossover included with NinjaTrader with nothing else running. Can the behavior still be reproduced?

[Shinjitsu noDeshi]

Hello Shinjitsu noDeshi,

This is caused by OneDrive, or other cloud syncing services or antivirus software, or could be from a custom coded script.

Please close all open workspaces, then in a new blank workspace open a new 1 minute chart with no template applied and do not apply any indicators.

Test the SampleMACrossover included with NinjaTrader with nothing else running. Can the behavior still be reproduced?

1) I'm not using any cloud syncing services, and the only AV software running is Microsoft Security Essentials, so that leaves the custom coded script or NT itself.

2) Windows Event Viewer shows no events at the time of the NT lockup.

3) This has only ever happened when data is streaming from IB and the current bar is actively updating. Meaning, it has only happened after historical bars have been loaded and tick data has been piped in for awhile (and is still being delivered). So if it's something with my script, then it would seem to be associated with OnBarUpdate. (I'm only overriding OnStateChange and OnBarUpdate.)

4) I can try with a 1m chart and the SimpleMACrossover script, but I'm not sure if I'll be able to reproduce the issue. The issue happens often, and appears to happen after ~24hrs of running NT, but I haven't figured out steps to reproduce it.

// Edit

5) Here's the pertinent portion from the trace file where the exception occurs.

2023-04-19 09:49:08:479 *************** unhandled exception trapped ***************
2023-04-19 09:49:08:479 Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock.
2023-04-19 09:49:08:828 System.Threading.LockRecursionException: Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock.
   at System.Threading.ReaderWriterLockSlim.TryEnterWriteLockCore(TimeoutTracker timeout)
   at System.Threading.ReaderWriterLockSlim.TryEnterWriteLock(TimeoutTracker timeout)
   at NinjaTrader.Data.BarsSeries.Dispose()
   at NinjaTrader.Data.Bars.Dispose(Boolean disposing)
   at NinjaTrader.Gui.Chart.ChartControl.<>c__DisplayClass595_0.<RemoveBars>b__0()
   at System.Windows.Threading.Dispatcher.Invoke(Action callback, DispatcherPriority priority, CancellationToken cancellationToken, TimeSpan timeout)
   at System.Windows.Threading.Dispatcher.Invoke(Action callback)
   at NinjaTrader.Gui.Chart.ChartControl.RemoveBars(ChartBars chartBars)
   at NinjaTrader.Gui.Chart.ChartControl.RemoveBarsFromBarsArray(ChartBars chartBars)
   at NinjaTrader.Gui.Chart.ChartControl.RemovePanel(ChartPanel panel, Boolean notifyUi)
   at NinjaTrader.Gui.Chart.ChartControl.Cleanup()
   at NinjaTrader.Gui.Chart.ChartTab.Cleanup()
   at NinjaTrader.Gui.Tools.NTWindow.CleanTabs(TabControl tabControl, Int32 idx)
   at NinjaTrader.Gui.Tools.NTWindow.OnClosed(EventArgs e)
   at System.Windows.Window.WmDestroy()
   at System.Windows.Window.WindowFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at System.Windows.Interop.HwndSource.PublicHooksFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
2023-04-19 09:49:09:042 Shutting down NinjaTrader
​

If I'm reading this correctly, the exception is being thrown when the window is being closed - but the problem occurs long before I've started closing stuff on NT.

[QuantKey_Bruce]

1) I'm not using any cloud syncing services, and the only AV software running is Microsoft Security Essentials, so that leaves the custom coded script or NT itself.

2) Windows Event Viewer shows no events at the time of the NT lockup.

3) This has only ever happened when data is streaming from IB and the current bar is actively updating. Meaning, it has only happened after historical bars have been loaded and tick data has been piped in for awhile (and is still being delivered). So if it's something with my script, then it would seem to be associated with OnBarUpdate. (I'm only overriding OnStateChange and OnBarUpdate.)

4) I can try with a 1m chart and the SimpleMACrossover script, but I'm not sure if I'll be able to reproduce the issue. The issue happens often, and appears to happen after ~24hrs of running NT, but I haven't figured out steps to reproduce it.

Zeroing in on the last thing you've said, that it happens after 24 hours, and understanding IB disconnects you every 24 hours, does it happen if you disconnect and reconnect a bunch of times?

[Shinjitsu noDeshi]

This has been reported a lot.

You might want to read (these are just the first few... there are more...)
https://forum.ninjatrader.com/forum/...ite-lock-error
https://forum.ninjatrader.com/forum/...rsion-8-0-27-1
https://forum.ninjatrader.com/forum/...-ninjatrader-8
https://forum.ninjatrader.com/forum/...k-error

That's... discouraging. Fortunately I haven't dropped any money to purchase NT.

It appears there's a threading issue in the NT core...

[Shinjitsu noDeshi]

Zeroing in on the last thing you've said, that it happens after 24 hours, and understanding IB disconnects you every 24 hours, does it happen if you disconnect and reconnect a bunch of times?

I'll check that. (I typically only have one disconnect/reconnect per day, when futures markets close.)

[NinjaTrader_ChelseaB]

Hello Shinjitsu noDeshi,

I am not aware of a specific threading issue in the NinjaTrader Platform.

If you comment the code out in OnBarUpdate() you are no longer able to reproduce the behavior?

[NinjaTrader_ChelseaB]

Hello Shinjitsu noDeshi,

Are the order submission calls causing the behavior?

If the logic is uncommented but order methods are commented out can you reproduce?

Are you able to reproduce with the Playback connection?

Are you able to reproduce with the SampleMACrossover?

[Shinjitsu noDeshi]

Zeroing in on the last thing you've said, that it happens after 24 hours, and understanding IB disconnects you every 24 hours, does it happen if you disconnect and reconnect a bunch of times?

Doesn't appear to. I tried manually a handful of times, and IB likes to send a number of really fast disconnect notices when it automatically re-logs in. Neither set of events caused the issue.

[Shinjitsu noDeshi]

Hello Shinjitsu noDeshi,

Are the order submission calls causing the behavior?

If the logic is uncommented but order methods are commented out can you reproduce?

Are you able to reproduce with the Playback connection?

Are you able to reproduce with the SampleMACrossover?

I don't have any order code in place - just code to mark up the chart, and that's all localized to OnBarUpdate. I've moved my initialization logic from OnStateChange (Configure) to OnBarUpdate.

I'll test the above as soon as I can, but I'm currently actively developing/testing/using this strategy for manual trading.

[NinjaTrader_ChelseaB]

Hello Shinjitsu noDeshi,

Reduce the code to the specific lines of code that reproduce.

Then isolate that code into a new test script and provide an export of this for me to test.

To export a NinjaTrader 8 NinjaScript so this can be shared and imported by the recipient do the following:

1. Click Tools -> Export -> NinjaScript Add-on...
2. Click the 'add' link -> check the box(es) for the script(s) and reference(s) you want to include
3. Click the 'Export' button
4. Enter a unique name for the file in the value for 'File name:'
5. Choose a save location -> click Save
6. Click OK to clear the export location message

By default your exported file will be in the following location:

• (My) Documents/NinjaTrader 8/bin/Custom/ExportNinjaScript/<export_file_name.zip>

Below is a link to the help guide on Exporting NinjaScripts.
http://ninjatrader.com/support/helpG...-us/export.htm

Once exported, please attach the file as an attachment to your reply.