Out of memory exception during backtest

[poseidon_sthlm]

Hi!
I have now migrated all my NT scripts to NT7. During the seccond backtest of the same strategy (1 year of 1 min data) it was aborted by the system and I was asked to send you my trace file. (The first backtest was succesful.)

2010-04-22 16:20:14:968 in OnUnhandledThreadException
2010-04-22 16:20:15:609 ********* exception trapped *********
2010-04-22 16:20:15:609 An exception of type System.OutOfMemoryException occured.
2010-04-22 16:20:15:609    on System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
  on System.Text.StringBuilder..ctor(String value, Int32 startIndex, Int32 length, Int32 capacity)
   on System.Windows.Forms.Control.get_WindowText()
   on System.Windows.Forms.TextBoxBase.get_WindowText()
   on System.Windows.Forms.Control.set_WindowText(String value)
   on System.Windows.Forms.TextBoxBase.set_WindowText(String value)
   on System.Windows.Forms.Control.set_Text(String value)
   on System.Windows.Forms.TextBoxBase.set_Text(String value)
   on System.Windows.Forms.TextBox.set_Text(String value)
   on NinjaTrader.Code.OutputWindow.OnTimerTick(Object sender, EventArgs e)
   on System.Windows.Forms.Timer.OnTick(EventArgs e)
   on System.Windows.Forms.Timer.TimerNativeWindow.WndProc(Message& m)
   on System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Why does this happen?

/Regards

[NinjaTrader_Bertrand]

poseidon_sthlm, are you using any custom resources in the strategies ported? Then please be sure to release those in OnTermination() to properly clean up after running them.

http://www.ninjatrader.com/support/helpGuides/nt7/ontermination.htm

[poseidon_sthlm]

Thankyou for your prompt reply, as usual.

I do use some rather advances custom indicators in my strategy. ( I have not programmed them myself but the code is open and doesn't need any .dll). The indicator uses a "Script Utility" that is a added in the top of the indicator with the keyword "using Script.Utility". This ScriptUtilty is another NT script that contains custom method for this particular indicator.

Unfortunately I'm not an advanced progammer and I'm not sure what you mean by :
"Then please be sure to release those in OnTermination() to properly clean up after running them. "

I have read about the new methods OnStartUp() and OnTermination() and they seam rather straight forward. But what should I put in these methods? The Indicator iiself? Could you point me to some code sample on how they should be used?

I'v noticed that you can't add an indicator in the initialize method in NT7 anymore like you could in NT6.5.

Variables
{
    MyCustomIndicator myIndicator;
}

Initialize
{
     myIndicator = MyCustomIndicator(x, y, z);
}

OnBarUpdate
{
     x = myIndicator.Values[0][0] + myIndicator.Values[1][0];
}

Can you do the same in OnStartUp() in NT7?

/Regards

[NinjaTrader_JoshP]

poseidon_sthlm,

Yes, the Initialize() line of code there would go into OnStartUp().

For OnTermination() you would need to release your resources there. Unfortunately I would not know what resources the ScriptUtility utilizes. Those resources would probably need to be released over directly from those scripts. Usually objects like if you used a StreamWriter or something like that would need to be released with the .Dispose() method of that object. Best may be to inquiry with the original author of the utility you are using.

[poseidon_sthlm]

Hi again,

It seems as it is this advanced custom indicator that causes the out-of-memory-exception when using the strategy analyzer. (This indicator uses Dictionary<int, IDataSeries> and Dictionary<int, DataSeries>) During optimizing I got this error message after the third run:

2010-04-23 13:11:30:500 ERROR: Error on calling 'OnBarUpdate' method for indicator 'MyIndicator' on bar 93695: An exception of type System.OutOfMemoryException occured.

I found this code in the NT 6.5 documentation

public override void Dispose() 
{ 
    // Clean up your  resources here 
    base.Dispose(); 
}

If my indicator is called MyIndicator how can celan it up with the Dispose method in OnTermination()? What code do I need to add in the OnTermination() method to release and clean up this indicator in my strategy before the optimizer is running its next loop?

/Regards

[NinjaTrader_Bertrand]

poseidon_sthlm, as Josh mentioned you would need to call the Dispose() method of your used custom resource object in the OnTermination() method to clean up properly after usage. Unfortunately we would not be familiar with the custom utility you employ.

[poseidon_sthlm]

Hi!
I have now checked the code of this indicator referenced in the error message and although this indicator probably is computational intensive, it doesn't use any unmanaged resources and doesn't open a file handle, allocate memory or open a database connection etc. It just calls some custom methods in another "Utility script". No strange code is used. One thing that I haven't mentioned earlier is that I actually don't use the indicator referenced in the error message direcltly in the strategy, but custom indicators that reference the indicator in question. (Indicator on indicator)

What we have found out this far is that:
1. the out of memory exception occurs after rerunning a simple backtest of my strategy more than twice

2. the out of memory exception occurs during optimizing my strategy on the third "loop" (which should be the same problem as 1).

3. the error message references a specific custom indicator

4. backtesting and optimization with one variable of this strategy with the custom indicator did work fine in NT6.5, although it was time consuming.

So this would point to that this is a NT7 specific probelm, doesn't it?
Since no unmanaged resources are used, the Dipsose() method wouldn't be relevant in this case, would it? (I'v spent a lot of time reading about IDsiposable, Dispose(), Finalize and Destructors etc. )
The memory consuming resource is obviously the indicator itself and the strategy analyzer doesn't free sufficient memory after each loop. So how can I help the strategy analyser to releese this indicator at the end of a loop?

Any advice on how I can go on with trouble shooting this problem?

/Regards

[NinjaTrader_JoshP]

Suggest you simplify your test scenario and find the exact indicator and method call in question. Then you will need to use some Print()s to try and see where in the indicator it starts chewing up your memory.