"Not enough quota is available..." followed by "Write lock..." error

[karlmac]

We have some indicators that we developed a couple of years ago and recently, we are getting this error message randomly when trying to run the indicators in at least 2 or 3 charts. This occurs especially when connected to a simulated/demo feed but not so much or not at all in a live feed. Below are some lines from our trace file.

The chart window freezes and sometimes it displays an error message saying "Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock."

The entire NinjaTrader client eventually hangs and I can only terminate it in the Task Manager.

I hope you can help.

2020-05-14 06:04:35:450 NinjaTrader.Gui.ControlCenter.OnConnectionStatus: Status=Connected
2020-05-14 06:04:35:450 (Simulated Data Feed) Cbi.Connection.Connect9 ok
2020-05-14 06:04:36:180 (Simulated Data Feed) Cbi.Account.AccountItemUpdateCallback: account='Sim101' accountItem=CashValue currency=UsDollar value=*****
2020-05-14 06:04:36:183 (Simulated Data Feed) Cbi.Account.OnTimerTick.Simulator.Roll: account='Sim101' fcm='' fcmDate='2020-05-13'
2020-05-14 06:05:35:297 Cbi.VendorLicense.Verify.1: trying 'https://license.ninjatrader.com/vendorlicensenew/Q***.xml' ...
2020-05-14 06:05:36:353 Cbi.VendorLicense.Verify.1: good
2020-05-14 06:10:02:151 *************** unhandled exception trapped ***************
2020-05-14 06:10:02:151 Not enough quota is available to process this command
2020-05-14 06:10:02:174 System.ComponentModel.Win32Exception (0x80004005): Not enough quota is available to process this command
   at System.Windows.Interop.HwndTarget.UpdateWindowSettings(Boolean enableRenderTarget, Nullable`1 channelSet)
   at System.Windows.Interop.HwndTarget.UpdateWindowPos(IntPtr lParam)
   at System.Windows.Interop.HwndTarget.HandleMessage(WindowMessage msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Interop.HwndSource.HwndTargetFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
2020-05-14 06:10:18:186 *************** unhandled exception trapped ***************
2020-05-14 06:10:18:186 Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock.
2020-05-14 06:10:18:193 System.Threading.LockRecursionException: Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock.
   at System.Threading.ReaderWriterLockSlim.TryEnterWriteLockCore(TimeoutTracker timeout)
   at System.Threading.ReaderWriterLockSlim.TryEnterWriteLock(TimeoutTracker timeout)
   at NinjaTrader.Data.BarsSeries.Dispose()
   at NinjaTrader.Data.Bars.Dispose(Boolean disposing)
   at NinjaTrader.Data.Bars.Dispose()
   at NinjaTrader.NinjaScript.NinjaScriptBase.SetState(State state)
   at NinjaTrader.Gui.NinjaScript.IndicatorRenderBase.SetState(State state)
   at NinjaTrader.Gui.Chart.ChartControl.Cleanup()
   at NinjaTrader.Gui.Chart.ChartTab.Cleanup()
   at NinjaTrader.Gui.Tools.NTWindow.CleanTabs(TabControl tabControl, Int32 idx)
   at NinjaTrader.Gui.Tools.NTWindow.OnClosed(EventArgs e)
   at System.Windows.Window.WmDestroy()
   at System.Windows.Window.WindowFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at System.Windows.Interop.HwndSource.PublicHooksFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)

[karlmac]

I was able to replicate this issue with this very simple indicator. Also attached is the trace file of the session before the NT8 client hanged.

Here's how I replicated it:

- Connect to Simulated Data Feed
- Open a chart of EURUSD
- Add the attached indicator
- Duplicate the chart in a new window around 4 to 5 times
- Change the timeframes of the chart windows and pan the charts repeatedly until one of the charts freezes
- Attempting to close a chart may sometimes display the error - "Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock."

EDIT: I removed my trace file as it may display sensitive info.

[NinjaTrader_Jim]

Hello karlmac,

Thanks for your post.

These two issues may be involved hand-in-hand. It would be good to see if resources are being consumed with your indicators that may be contributing to a memory/quota related issue, and also to see if your indicators are producing a dead lock scenario separately from the quota issue.

One might be causing the other. For example, if the indicators are cresting a dealock scenario, it could lead to a situation where resources are not freed and then a quota related error would come up.

If you are performing asynchronous operations in your scripts that would be a good first place to look for issues.

You can also increase virtual memory to help alleviate quota related issues. Many people on the internet recommend setting virtual memory to be 1.5x that of physical memory. This may hide an underlying issue, but can still help to rule out if quota related issues are causing a deadlock scenario. If you still get hanging symptoms after increasing virtual memory, then this would point to a locking issue involving your script.

If there is a locking issue using standard NinjaTrader components without custom asynchronous code added, we would like you to send us a reduced example (preferably, if possible) that can reproduce the locking issue.

Below is are some publicly available resources for changing virtual memory.

https://support.microsoft.com/en-us/...er-performance
https://www.x-plane.com/kb/increasin...ry-on-windows/


EDIT: I tested the attached script following your steps and I did not hit this issue. Do you see the issue after changing your Virtual Memory settings and restarting the computer?

We look forward to assisting.

[karlmac]

Many thanks for testing my script, Jim. I will attempt to increase my virtual memory and test this again. Perhaps it is worth mentioning that I run 4 internet browsers and some other programs alongside NT8 but my RAM is 16GB. I'll let you know if I can still replicate the error after these adjustments.

[karlmac]

Hi Jim,

I followed your advice and increased the virtual memory of my computer to 1.5x its physical memory but I was still able to replicate the issue twice. I recorded these in the following animated GIFs below. Only NinjaTrader 8 was running (aside from my screen recorder) and I had plenty of memory to spare - 16GB.

Replicating write lock error: https://i.imgur.com/BjlMAzX.mp4
NinjaTrader 8 hangs while running small sample code: https://i.imgur.com/P8RgVaa.mp4

I hope this helps.

EDIT: Please ignore the attached link below it doesn't work

[-=Edge=-]

Hi Jim, I followed your advice and increased the virtual memory of my computer to 1.5x its physical memory but I was still able to replicate the issue twice. I recorded these in the following animated GIFs below. Only NinjaTrader 8 was running (aside from my screen recorder) and I had plenty of memory to spare - 16GB.

Your problem is in your State == State.Configure.. The BarsPeriod and BarsPeriodType should not be accessed in State.StateConfig

NinjaTrader 8 Help Guide

https://ninjatrader.com/support/helpGuides/nt8/?barsperiod.htm

[karlmac]

Many thanks for the tip Edge! I'll try that one.

[NinjaTrader_Jim]

Hello karlmac,

-=Edge=- is correct that it is unsupported to use items that are dependent on the data being loaded in State.Configure. (See AddDataSeries documentation for more details.)

Are you able to reproduce if you use the following overload instead?

AddDataSeries(string instrumentName)

AddDataSeries() - https://ninjatrader.com/support/help...dataseries.htm

I look forward to hearing back.

[karlmac]

Hi Jim,

I edited my script so it follows this overload - AddDataSeries(string instrumentName)

I commented out any references to BarsPeriod and BarsPeriodType in my code and it hardly does anything but load 7 data series but I was still able to reproduce the issue as shown in the GIFs below.

https://imgur.com/O356YBm

https://imgur.com/a/q51fyU9

- Karl

[koganam]

I was able to replicate this issue with this very simple indicator. Also attached is the trace file of the session before the NT8 client hanged.

Here's how I replicated it:

- Connect to Simulated Data Feed
- Open a chart of EURUSD
- Add the attached indicator
- Duplicate the chart in a new window around 4 to 5 times
- Change the timeframes of the chart windows and pan the charts repeatedly until one of the charts freezes
- Attempting to close a chart may sometimes display the error - "Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock."

EDIT: I removed my trace file as it may display sensitive info.

Thank you for this. This issue has bugged me endlessly, but I have never been able to reproduce it with a simple indicator. It always shows up only when I am running many copies of a complex indicator that is performing statistical analysis on multiple instruments.

Unfortunately, NT8, I am reasonably sure is entirely generated by Visual Studio without hand optimization, so there may be little that any of us can do. It turns out that this message about quota exhaustion is almost entirely due to a Windows Messaging Queue limitation of 10k messages. That looks like a lot, until you meet an application (framework) which is flooding the queue with messages. Remember how you are inducing it by moving charts around somewhat rapidly?

There is a registry key, USERPostMessageLimit, that can be used to adjust this limit. Somehow I doubt this is a viable workaround. WPF rendering just sends a whole lot of messages. Add all the stuff we (too ambitiously?) try to do with NT8, and we all have problems. I would not even blame it on NT8. This looks more like a Windows issue.

ref: https://docs.microsoft.com/en-us/win...threadmessagea

[karlmac]

Many thanks koganam for sharing and it's good to know that I'm not alone on this one. I can replicate this error much easily in my real indicator where there are much more lines of code involved.

[NinjaTrader_Jim]

Hello karlmac,

Could you write in to platformsupport [at] ninjatrader [dot] com with the text "Attn Jim 2555747" and a link to this thread? I would also like your log and trace files and a copy of the indicator that is more likely to hit the issue.

The issue sounds to be more environment related, but I would like to test on our end using Virtual Machines so we can try to hit the same and investigate further.

Attaching Log and Trace files

Please follow the steps below to manually attach your log and trace files to your email so we can investigate further..

• Open your NinjaTrader folder under My Documents.
• Right click on the 'log' and 'trace' folders and select Send To> Compressed (zipped) Folder.
• Send the 2 compressed folders as attachments to an email to platformsupport [at] ninjatrader [dot] com
• Once complete, you can delete these compressed folders.

I look forward to assisting.

[karlmac]

Many thanks Jim. I have sent what you requested.

[AnkaS]

Dear NT Support,
Am facing this problem in NT version 8.0.22.2 64-bit. Any final recommendation on how to avoid this error ie - increasing virtual memory size, increasing USERPostMessageLimit etc.

Regards

Vivek

[NinjaTrader_Jim]

Hello Vivek,

The latest version is Release 23.2, but I do not think the version of NinjaTrader is the source of the issue.

Update - https://ninjatrader.com/PlatformDirect

We have not been able to reproduce symptoms on our end using the example karlmac presented.

We do tend to see this if several error message popups are created, which could be something that could be addressed in a script if the script is causing the pop ups to be created.

Increasing Virtual Memory has been known to help avoid quota related errors, and Koganam's input for adjusting USERPostMessageLimit may also be worth trying. It will be helpful to the thread to report back if adjusting USERPostMessageLimit helps in your case.