Strategy analyzer problem, backtests not running

[adamus]

Hi,

I have a problem running backtests in the Strategy Analyzer. At some point soon after starting and doing a couple of backtests, the strategy analyzer refuses to run anymore.

I think it is linked to compiling the script.

There is the following message which appears in the trace file more or less at the same time as the problem arises:

2010-06-04 00:43:02:921 Code.Editor.OnLoad: Failed to reference 'NinjaTrader.Custom, Version=7.0.0.16, Culture=neutral, PublicKeyToken=null': System.IO.FileNotFoundException: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
at System.Reflection.Assembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
at System.Reflection.Assembly.InternalLoad(AssemblyNa me assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, StackCrawlMark& stackMark)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at NinjaTrader.Code.Editor.OnLoad(Object sender, EventArgs e)

[NinjaTrader_Jason]

Hello adamus,

Can you please post your entire trace file. If it is too large, please zip it first and check if you can post it. If not, please let me know the size of the trace file.

You suspect it is related to compiling the script. Can you please clarify. Do you compile your NinjaScript files while runing a backtest?

What happens when the Strategy Analyzer refuses to run - do you receive an error message?

[adamus]

I do the following when I get this error:

I open NT.

I disconnect from the data supplier which automatically connected - I have years of historical data in the HDM.

I open a strategy analyzer, choose my strategy, my instrument and my test period, and set the parameters for the backtest.

I run the backtest.

Strategy analyzer gives me the results. I always check the chart in the performance results - which could be the problem.

I open the strategy script and also the User Defined Methods, change script and compile.

I go back to the Strategy Analyzer and hit 'backtest'. The old performance results disappear and come back within 2 or 3 seconds, but without all the tabs - just the overview, the chart and the settings tabs still there. The settings tabs are the old settings and instrument, not the new one. Also none of the debug print statements from the script are output to the output window.

I don't compile while the backtest is running - it's only a 30 second backtest.

I don't see any error message, or any error in the log. Only what I posted from the trace file.

If I close NT and re-open it, I can run the backtest again.

I only noticed this yesterday but I suspect it may have happened earlier. I upgraded to NT7-b16 last week or the week before and had some problems with the code libraries not being referenced, because the new install of NT7 went into a different directory and the libraries were hardwired to the old directory from the scripts. Maybe that's relevant somehow still?

[NinjaTrader_Jason]

Can you please tell me when exactly you cannot run a backtest anymore. Is this after you backtested the strategy after the modifications you performed?

Could you provide me with the strategy and tell me what exact modifications you performed, so I can test it on my end.

[adamus]

OK a bit more testing shows that it depends on compilation of the User Defined Methods.

If I compile that and then try another backtest, I get the problem.

EDIT: it doesn't seem to matter whether the strategy I'm trying to run uses the methods in the User Defined Methods or not. A simple test strategy is displaying the same issue.

[NinjaTrader_JoshP]

adamus,

The Code.Editor.OnLoad message you saw in your traces can be effectively ignored.

When you are viewing the backtest are you sure it is simply not trying to calculate something prior to you shutting down NT? Depending on the tab you are on it may take some time to populate in all the information. Please use Windows Task Manager and see if NT is processing things either on CPU or memory.

This is what I tried on my end and it worked.
1. Open Strategy Analyzer
2. Backtest SampleMACrossOver
3. Open UserDefinedMethods (relatively blank file)
4. F5 to compile
5. Backtest same strategy again --> no issue

Please try with a default UserDefinedMethods file with no methods inside and see if you can reproduce. Thanks.

[adamus]

Josh,

I took all the user defined methods out of the User Defined Methods script file and put them in the strategy script.

Then the issue with recompiling User Defined Methods was solved, and the same issue arose for recompiling the strategy script on its own.

I think I have isolated it now. Attached is a strategy script which produces the issue when recompiled:

- put the script in your strategies folder

- open a new strategy analyzer

- open the strategy from the strategies folder on the left

- compile the strategy

- run a backtest - I think any instrument, period, timeframe etc will do, but make it a short period because this test script trades on every bar

- recompile the strategy

- try to run a backtest - it fails and redisplays the previous results.

[NinjaTrader_JoshP]

adamus,

Your code does not compile on line 51. Please remove all your custom methods and only use NT ones.

[adamus]

Line 51 shouldn't be there. Please delete it and try again.

[NinjaTrader_Jason]

Hello adamus,

I tested the strategy on my end using the steps you outlined, however I did not experience the backtest failure - it did display the results of the second backtest as expected.

I backtested the strategy first on 10 minute bars on the ES 06-10 from 6/1/2010 to 6/6/2010. I compiled the strategy and subsequently ran the strategy on 5 minute bars on the same instrument and amount of historical data. It displayed different results as expected.

I left the code window opened during the first backtest. Once finished I used the code window to compile the strategy for a second time. Do you perhaps use different steps to compile your strategy a second time?

Does the issue occur when you do not compile your strategy a second time in between backtests? Does the issue occur when you compile the strategy as per the steps I described above?

[adamus]

I keep code windows open the whole time.

You did what I do. After the first backtest, I just switch windows to the strategy script, change something and recompile, and then it won't backtest again in that strategy analyzer window.

If I don't compile the strategy twice with a backtest in-between, it doesn't cause the issue.

I guess it must be something to do with my windows installation and the System.IO library, doing a bad compile or something.

What a great opportunity to rant about how rubbish windows is. I only bought windows so that I could run NinjaTrader. This issue, which will probably disappear like a few of the other issues I've had, is so typical of windows in comparison with Mac and linux, it's not good. You guys should write NT8 in Java.

[NinjaTrader_Jason]

Can you please give me an example of what you changed in the strategy code, so I can test this as well. I did not make any changes in the strategy code when I tested it.

Does the issue occur when you compile your strategy without a change in between the backtests?

[adamus]

Yes. I slipped accidentally out of 'bug define' mode there - you don't need to change anything, just recompiling will cause the error.

[NinjaTrader_Jason]

I tested the steps once more, however I do not experience any issue on my end.

[adamus]

Shame. Another mark on the 'not reproducible' list then.