Replay out of memory

[ATI user]

Please explain what Market Replay does as far as memory requirements are concerned...i.e. what is it keeping track of that accumulates to cause the out of memory condition?

I note that on rewind it takes a very long time (3-4 minutes) for Replay101 to reset...so is the problem the trade history that builds up?

I am trading the ES continuously for 10 days...code places 512 atm trades plus maybe another 800 atm trades scaled in...then I get this message on the same day at the same time twice in a row now

I only have 1 workspace with 2 charts ..one chart has charttrader and a very simple indicator running and the other chart has the strategy running

Log file is only 4 megs...Trace file only 20 megs...NinjaTrader.sdf is only 192 megs....I have 3 gigs of RAM and nothing else running other than Firefox

where can I locate the out of memory condition?

[NinjaTrader_Jason]

Hello ATI user,

The error message indicates NinjaTrader had to use more memory at that moment than available. As far as I know nothing is accumulated memory wise when using the Market Replay connection.

It can take a while for all data to be loaded when you have continuous Market Replay data for 10 days.

Processing all these orders at once seems too much to handle if the error message recurs each time at that moment.

I suggest to check if the issue recurs when the chart with ChartTrader and the indicator are shut down. Please check if replaying less than 10 days at a time will avoid the error as well.

[ATI user]

Hi Jason

Thanks. I will try with only strat chart running and Output Window and Firefox and all other applications closed.

What do you mean by 'Processing all these orders at once seems too much to handle if the error message recurs each time at that moment.' since there are only a max of 6 orders in play at one time. Is it because Charttrader has to keep track of them all?

Also what do you mean by processing less than 10 days data? It runs fine up the the one point in time so less than 10 days works until then. I am not retaining any info in the strat that would accumulate...i.e. only dealing with 'in the present' data to make trading decisions and move stops

[NinjaTrader_Jason]

Sorry, I misinterpreted and assumed that many orders were submitted at once.

Once an order is executed, it is saved in NinjaTrader. I am not sure if this is causing the error. Do you notice any consistency when the error occurs - it only occurs when so much orders have been submitted?

You mentioned you replay 10 days continuously. I suggest to check if the error occurs when replaying only 5 or 8 days using the From and To dates.

Or are you using From and To dates that span greater than 10 days and you just ran the strategy on 10 random consecutive days? Please check what happens when you use From and To dates that only span the dates you like to test.

[ATI user]

orders are submitted as follows:
1 atm on trigger condition
1 scalein on profit condition
2 etc
3 etc
4 etc
to a maximum of 5 orders to maintain
the code trails the stop on the intiial trigger atm lots
the others trail auto per the atm

on condition, the orders that have not stop out are closed and position is reversed.

re days, the goal is to let it run indefinitely until it finally drawdowns enough to stop the strat

anyway..I took your suggetion and ran with Charttrader chart closed and no problem..let it run and strat closed out and shut down as expected...it had run from 5 aug to 17 aug ....MR data then ran on to end of data 30 sep. ...no memory issues...perfect....so you nailed it again Josh..thanks

question now would be...what does Charttrader keep track of that would cause the memory issue? maybe all those plots it likes to do? which I never use

[NinjaTrader_Jason]

Good to hear the error did not persist after closing the ChartTrader chart.

Running a chart with an indicator applied while connected to the Market Replay connection will take up memory. Accumulated with the memory used by the strategy, it crashed the software. Since you exlcuded the chart, it reduced the memory usage enough for the error not to occur again.

[ATI user]

I will have to test without the indicator to see for sure. All it does is calculate the current session range and plot it for the current session and for the prior session on close. So....not very demanding on memory.

Charttrader must be more demanding on memory I would think. Will let you know.

[sledge]

I will have to test without the indicator to see for sure. All it does is calculate the current session range and plot it for the current session and for the prior session on close. So....not very demanding on memory.

Charttrader must be more demanding on memory I would think. Will let you know.

What is windows task manager showing?

chech your remaining harddrive space.

[ATI user]

Good to hear the error did not persist after closing the ChartTrader chart.

Running a chart with an indicator applied while connected to the Market Replay connection will take up memory. Accumulated with the memory used by the strategy, it crashed the software. Since you exlcuded the chart, it reduced the memory usage enough for the error not to occur again.

ok...ran fine with Charttrader chart running without the indicator...that would indicate the indicator was the problem I assume

the only other variable is that I had Firefox running before....will see if I have any other problems

[ATI user]

What is windows task manager showing?

chech your remaining harddrive space.

assuming NT is using the C drive, I have 327 gigs of free space ...did not check when NT froze..will next time...I just assumed it could not be a space issue but rather NT getting hung up trying to do too much at the same time

[ATI user]

ran a test which crashed again...see pic of memory usage attached....had charttrader chart running with no indicator and firefox

computer is a quad core...does NT use all 4 cores in replay Josh? It would appear so, but maybe that is the problem?

ran a second test with nothing running except NT with the strategy chart...ran fine ...ran out of ES 09-11 contract instead of memory....ran from 5 aug through 7 sep....

[sledge]

assuming NT is using the C drive, I have 327 gigs of free space ...did not check when NT froze..will next time...I just assumed it could not be a space issue but rather NT getting hung up trying to do too much at the same time

With 327 gigs, you are fine.

I've been down to a hundred megs before NT had out of memory because my harddrive filled up.

[NinjaTrader_Jason]

Hello ATI user,

NinjaTrader 7 has optimized multi core/multi-processor support. This would allow you to make full usage of multi-core/ multi-processor hardware. However, due to the nature of the Window OS, NinjaTrader 7 would not be able to fully use all available multi-cores/multi-processors at any given time.

[ATI user]

It seems to me that replay stores a lot of history that is causing the problem for long replays.

I notice each time I rewind and replay, it runs slower. Also it takes a long time to reset Replay101 and clear the history. For instance a day that makes 10 trades will rewind in a few seconds whereas a day makes 500 trades will take 3-5 minutes to rewind.

I would like to use replay without any history being stored. It is a waste of time and sometimes causes NT to crash. This would be a good option in my opinion.

In the meantime, what is replay storing and where? maybe I could just delete that file prior to each rewind?

[NinjaTrader_Jason]

I will forward your suggestion to not store trade data when running the Market Replay connection.

Trade data is stored in the database. Unfortunately it is not supported to alter data in the database.