Constant crashes on optimizations

[molecool]

I have reported on this before but am disappointed to see that the optimization crashes persist in NT7 b20. I am testing the following permutations:

http://screencast.com/t/ODg5YTYzMz

Crashes every time - just freezes and memory increases to over 4GB.

I mean - this is ridiculous and you guys have to fix your memory management issues. This is a pretty modest sampling and if you want to sell your tools to pros then you have to offer more than a handful of parameter permutations for a moving average or otherwise trivial.

FYI - I'm running a very fast system - QuadCore Win7 64bit with 6GB of RAM. One year old but still considered top of the line.

I can't test any of my strategies without NT7 crashing - at this point I'm ready to give up and move my code base to TradingBlox or something more professional than this. I constantly feel that NT is the bottleneck as the data keeps getting corrupted, the test crashes, strange errors keep popping up, etc. I am spending most of my time walking on egg shells as to not crash NT and the quality of your codebase at least at this stage still leaves a lot to be desired. This is not a professional tool that I can continue advocating.

FYI - I run a trading blog that is frequented by over 50,000 traders per month. I have often praised your tool and given you free coverage as I like the idea and the especially the equidistant charting. I also like the general UI and implementation of NT. But I just can't keep spending my time on trying to do simple things as it is extremely unproductive and time intensive. I have spent the last two years trying to work with NT - first with 6.5 and now with 7.0 - both versions just don't cut it for anything beyond very simplistic strategies - sorry.

Molecool

[NinjaTrader_Austin]

Molecool, could you please either post the strategy here or email it to support at ninjatrader dot com referencing 'ticket 358426' so we can further test this?

[molecool]

I can't really post or email my strategy. If I can send you log/trace files that may help...

[NinjaTrader_Dierk]

Molecool,

I'm sorry you continue experiencing trouble using NT7. However, all memory leaks resulting from backtesting/optimizing custom NinjaScript sources which we researched so far had been a result of coding bugs in those sources. At the same time we are not aware of any inherent memory leaks regarding NS strategy backtesting/optimization. This is not to say that there actually could be an issue.
Thus, there is a significant likelyhood that the source of trouble you experience may be on your custom code. I'm not saying it actually is, I'm just pointing out probablitties.

So, here are your options as I see them:
- you send me your strategy for further research. I understand you consider this your sensible intellectual property (we would handle your strategy appropriately). However, we have gone this road in the past with other users/partners in the same situation without any complaint
or
- you strip down your strategy to the "bare essentials" to focus on the issue you experience and possibly strip out whatever you consider your IP and then provide us that "crippled" version of your strategy for further research

Please let me know how you wanted to proceed.

Thanks

PS: sending traces/logs would not help

[molecool]

Update

After installing NT7 b20 I feel that a revision of my prior quite passionate post is in order. And I also hope that my own personal observations may help others suffering from 'NinjaTrader fatigue' as well ;-)

The first thing I did was to install beta 20 as I wanted to be up to date. The second very simple fix that really made a whole world of a difference was to simply disable any log statement with a conditional in my strategies as such:

if (logEnabled) Log("This is my var: " + myvar, LogLevel.Information);

BTW, I even used a similar approach way back with log4j as the right side of the expression is being evaluated first after which the condition is checked. Which is a big waste of time. But as I see it there are no real log 'levels' in NT at this point. I guess I could use an outside package however (similar to log4j) and then I would still recommend using a condition as you don't want the log expression to be evaluated (this is assuming that C# executes right to left like Java).

The 'logEnabled' flag is something I added as a parameter in my own base class, something I greatly recommend. I used to code in Java a lot and really enjoyed log4j which allowed me to set my log levels via XML. That meant that only my current log levels were being printed (i.e. info/debug/error, etc). Maybe this is also possible in NT but again - if not I'm sure there are other log packages out there you can bind in.

Anyway, doing this returned any strategy that previously entered melt down mode a few percent into a comprehensive optimization session back to a stable. I for one use log statements quite liberally (that's my style) and since there is no debugger in NT (I know of) this is a good way of finding bugs. But turning your logs off when back testing or optimizing is key and will keep your memory in check. I'm currently running a pretty extended test and it's using 1.3GB of RAM. Before I turned off the logs it regularly shot above 3.5GB - after that it crashed.

But the watershed moment for me was when I signed up for Kinetick this weekend, thus replacing my IQFeed connection. The symbols are all identical, so migration is super easy. The feed thus far seems stable, however I have not seen it during trading hours - so I will report back if I see any problems.

The speed of Kinetick is awesome however and it seems to speed up my optimization sessions by at least 20% or so (that's not a real measured number - just a wild guess). IMHO the IQFeed/NinjaTrader interface was never a really good one to begin with and I'm glad to see that NinjaTrader took the initiative and provided a more direct connection. As I understand it this is a collaboration between DTN (i.e. IQFeed) and Ninjatrader - which I'm happy about as DTN has good customer service and beyond the dreaded interface issues never gave me many reasons to complain.

In short: I'm a pretty happy camper now. In the past few months I had almost but given up on NinjaTrader as almost every backtest resulted in a crash (in 6.5 and 7.0). The tables have completely turned now and I can even switch between several workspaces and create new charts while I'm optimizing!! That is a huge step forward and I feel a bit vindicated now as I have spent a lot of time and effort getting up to speed on NinjaScript.

Anyway, that's it. I complain a lot and I can be a pain in the ass. But I also give credit when it's due and I consider NT7 a huge step forward. If any of you guys have problems testing strategies look for bugs first. Then get rid of the log statements and finally you may sign up with Kinetick ;-)

Cheers,

Molecool

[NinjaTrader_Dierk]

Thanks for your feedback. Glad you got those issues squared away.

[molecool]

Squared away indeed - feels like driving a Porsche now. Very happy.