Questions regarding multi-day, multi instrument capability

[bridenour]

Hi,

I thought i would provide an update as to the outcome of the strategy i was developing. Basically, i developed a strategy with the following requirements:

1. strategy scans the full russel 1000 daily for signals
2. strategy takes buy signals off daily bars, and holds positoins for multiple days
3. strategy uses minute bars for exit signal processing
4. strategy requires about 55 day bars history to process correctly


I was able to create a strategy that worked functionally. I could load the ticker list from a file, add the minute and day bars manually, and manage a aggregate account balance for the entire portfolio of open positions. Functionally, everything worked fine.

Performance wise, i do not believe it would have presented much of a problem to process minute bars, even for the full 1000 instruments.


BUT, unfortunately, the strategy does not work properly when executed against all 1000 instruments. NT appears to load too much data into memory, and it eventually freezes before completing initialization. Basically, it uses up about 1.5 gigs of memory and then fails.

Now, this failure is happening in the backtest, i dont know yet if the same would happen during live testing, but i assume so. I am only attempting to back test about two months of data. I assume it is the minute data which is killing the system, which is unfortunate given that i really don't need any minute history since it is just used to exit the position.

If anyone can answer a questions about how NinjaTrader functions when running live -- is it going to aggregate and store in memory every piece of price info it receives? Or does it possibly flush/release the memory after the price data is beyond the horizon of the strategy?


I am currently in the process of determining where the break point is, in terms of number of instruments it can support. It actually fails during the backfilling of historic data, so I am holding out some minor hope that it will not act in the same way when running the strategy live.

Thanks,
Brandon

[bridenour]

Well, I'm super disappointed. First, with minute bars involved, the system just crashes when backfilling data if i have more than a couple of hundred of instruments. seems to be related to a memory limitation ( i have two gigs, it uses it all).


Disappointing, but i could have lived with that. What is worse is that when limiting the list to just 100 instruments, it still doesn't work. It does complete the historical data backfill (or, at least, it seems to). but, it never executes the strategy -- it just stops. It doesn't seem to get past initialization. I know this because i have a print statement at the beginning of the strategy, and it never gets printed.

Looking in the trace and elsewhere i can't find any error messaging -- the strategy just seems to stop executing once the backfill is done.

Any advice or guidance on this would be appreciated....Im about 100 hours in now and frustrated that something that works for 5 or 10 instruments fails so quickly when scaling.

thanks

[NinjaTrader_Ben]

Hello,

Someone will reply to your post on Tuesday. Thank you for your patience.

[bridenour]

Thanks


I discovered some addition info as to why this isn't working. There is definitely an upper bound memory limitation to the multiinstrument approach (at least on my computer), but i am having a secondary problem.

Some individual instruments are causing the backtest to not run (for example, LIFE is one). I basically, through trial and error, was able to identify that this security causes the strategy to never execute. Once i removed that strategy , the list of instruments processed. I then ran the strategy with just that instrument, and it also failed.

The problem is there is no error message whatsoever in the trace logs or TDA log. I'm pretty sure this is not the only security with a problem, and there is no way to tell which instruments are causing this failure and which are not (as far as i can tell). Although i was able to find this one, there is really no way i could go through all 500 and determine which instruments are problematic.

Thanks in advance

[bridenour]

Further progress in this grand mystery:

I discovered that life sciences did not exist before 11/24/2008. It was created on that date by the merger of two companies. Therefore, data for that instrument is not available prior to that date, and the requisite 55 historical bars are not available.

So, it appears if there is a single instrument in my list of 1000 that doesn't have 55 days of history, the entire strategy will not run. No error messages are displayed and there doesn't seem to be any way to determine which symbols are problematic. The only thing i can think of is handchecking each and every symbol to make sure it has adequate data. Painstaking, obviously, but it should work.

[NinjaTrader_Ben]

Hello,

Glad you were able to sort it out.

I do not know exactly what you are doing but maybe you could build a condition for each instrument BarInProgress and test to see if Count > 55. Perhaps throw up an alert if it is not, so you know. I haven't tested this, but may be worth a try.

[bridenour]

It doesnt ever execute anything past Initialize. You can recreate this even with a single instrument strategy backtest -- if there isn't enough backfill data available, the strategy won't execute but you don't get any information as to why not.

when dealing with large amounts of instruments, this can be very frustrating.

Thanks

[NinjaTrader_JoshP]

bridenour,

Depending on your BarsRequired property, you will need at least that many bars from each of your bar objects. You can try setting this to a lower number, but some indicators may be in an "unstable" period before that time.

[bridenour]

Thanks for the reply. I cant set it to less bars, otherwise the indicator isnt there that is needed to generate signals.

Anyway, my only suggestion is to have it message the user in some way when this occurs. To effectively run against any large number of instruments, this is really necessary.


One more question for you with regard to how NT manages memory. Lets say I run my strategy against 250 instruments and plan to leave it running over an extended period of days. Will it accumulate all pricing data for all instruments into memory? or will it release the pricing data that is beyond the "BarsRequired" horizon?

When i run a backtest on this many instruments, it will crash if i run it on a period longer than three months. However, the strategy itself only needs about 60 days of data, which is fine from a memory footprint standpoint. The question is, when i run it live will it keep a memory footprint that represents the rolling 60 days needed for the strategy, or will it perpetually aggregate based on the starting date?

Let me know if that question is unclear!

Thanks

[NinjaTrader_JoshP]

bridenour,

Thank you for your suggestion. I have forwarded it to development.

NT memory is managed by .NET garbage collector and is released whenever .NET decides to release it. There will be improvements in memory handling and performance in NT7.

[bridenour]

Josh,

Im not a C# expert, but I am going to assume that memory is reclaimed only once it is released by the application (and then is collected by the GC).

My question is really how and when the application releases price management during a strategy execution? If i start a strategy, does it retain all price data in memory for the duration of the execution, or will it free some of it after it ages to a certain point?

Thanks for your help!

br

[NinjaTrader_JoshP]

bridenour,

I am not sure and will have to confirm with development.

[bridenour]

Thanks for your help - im just wondering if the memory footprint will steadily increase once running. If so, it wouldn't be practical to run so many instruments in this type of strategy.

br

[NinjaTrader_Ray]

"Old" bars are not disposed of thus additional bars added will of course consume more memory.

[bridenour]

Ok, that answers my question -- thanks for the reply.

Obviously, it goes without saying that to use this with even a moderate amount of symbols over extended periods (multiple days), this very quickly becomes a problem. It looks like the product is more geared toward intraday tradng, but this is something to note depending on the future direction of the product.

Also, I notice when i backtest NT very quickly ramps up to consume over 1 gig of memory. What is also notable is that once the backtest is done and the strategy analyzer closed, the memory consumption stays at the same level. Seems to be a one way boulevard on the memory usage front.

Anyway, thanks for the info -- saves me the disappointment of discovering this behavior live...