Replay out of memory

[NinjaTrader_Brett]

Hello,

Thanks for the feedback will make sure it is forwarded to the correct place.

Thanks.

[ATI user]

ATI user,

Thanks for clarification.

I just went through a years worth of replay data and was able to reset back to the beginning without delay. How much replay data are you going through at once? What version of NinjaTrader do you have? How many charts do you have open at once? Please find some helpful performance tips at the following link.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?performance_tips2.htm

Please let me know if I may assist further.

I am running up to 3 months per replay of ES data generating 1000-1500 trades. This takes 3 or 4 or 5 hours to run....then takes 20-30 to reset on rewind

if you ran a year I would assume you are not making any trades? I am assuming part of the 'history' recorded during replay is for trades? ...and therefore takes time to delete?

I am running the latest NT version on a quad-core (I believe NT utilizes the additional core(s) during replay)....only one chart with the strategy....

[ATI user]

Hello,

To clarify, NT does a lot of undocumented stuff to reset when you rewind.

Including clearing buffers, unloading and loading replay files, checking replay file time and dates, reseting accounts. All this takes time and unfortunately there is no more specifics I could provide on this process at current. Other then we are adding more functionality and overhauling some new replay features in our next major release of the software that is still sometime down the road however. I do not have a date or can comment on specifics of features unfortunately.

If you want to do a manual reset on the Replay data this is all stored in the database. This would require either a database reset or rename of the database which you would be aware of most likely ATI how to do. If not I can provide instructions. Also, you can delete replay data that you no longer use in the My Documents->NinjaTrader 7->db->data folder. As you may have some corruption here or a file that is just taking way to long to access could be causing lengthy market replay connect and rewind time if database reset does not resolve.

TO further clarify the replay feature requires unaltered access to files. If you have anti virus or backup software that would try to scan these files at the same time NT is accessing this would cause a huge slow down in speed or non responsivness. Make sure the My Documents->NinjaTrader 7 folder is on the exclusion list for anti virus realtime protection and automatic backup software.


-Brett

thanks Brett...that is more what I was after

anti virus and backup are not issues...neither is corrupt data

so that leaves 'a lot of undocumented stuff to reset when you rewind.' ...which I assume is the 'History'

it would be great if this could be streamlined if not done away with completely...for instance as an option. I assume this is what you are working on in the new version.

As it stands, every day extra that I include in a replay, a lot of additional time is required to reset on rewind. I am doing replay on 3 computers simultaneously for ES data that can continuously replay as long as 3 months of data (the length of a futures contract)...so, collectively, the wasted time adds up and adds to my frustration as I will replay on and off 10 to 12 hours a day 7 days a week at times.

I look forward to the new release...however long it takes

[NinjaTrader_AdamP]

ATI user,

Thanks for clarification.

I just went through a years worth of replay data and was able to reset back to the beginning without delay. How much replay data are you going through at once? What version of NinjaTrader do you have? How many charts do you have open at once? Please find some helpful performance tips at the following link.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?performance_tips2.htm

Please let me know if I may assist further.

[NinjaTrader_Brett]

Hello,

To clarify, NT does a lot of undocumented stuff to reset when you rewind.

Including clearing buffers, unloading and loading replay files, checking replay file time and dates, reseting accounts. All this takes time and unfortunately there is no more specifics I could provide on this process at current. Other then we are adding more functionality and overhauling some new replay features in our next major release of the software that is still sometime down the road however. I do not have a date or can comment on specifics of features unfortunately.

If you want to do a manual reset on the Replay data this is all stored in the database. This would require either a database reset or rename of the database which you would be aware of most likely ATI how to do. If not I can provide instructions. Also, you can delete replay data that you no longer use in the My Documents->NinjaTrader 7->db->data folder. As you may have some corruption here or a file that is just taking way to long to access could be causing lengthy market replay connect and rewind time if database reset does not resolve.

TO further clarify the replay feature requires unaltered access to files. If you have anti virus or backup software that would try to scan these files at the same time NT is accessing this would cause a huge slow down in speed or non responsivness. Make sure the My Documents->NinjaTrader 7 folder is on the exclusion list for anti virus realtime protection and automatic backup software.


-Brett

[ATI user]

ATI user,

Thanks for your note.

The market replay data you download yourself through File > Utilities > Download Replay data or it was saved by having "Record for Market Replay" checked. This is tick data, so if you are downloading/saving a month/years worth of data it will most definitely be a large file, and may take some time to delete. It may be faster to manually delete it yourself in the Documents\NinjaTrader 7\db\data directory or set up some Windows scheduler that deletes this data every night when you are off your PC.

Please let me know if I may assist further.

let me try again

this has nothing at all to do with me trying to delete data files from my computer...nor how to download data

please read my posts... carefully..

this issue is when you are running market replay and then want to change the date in market replay, or rewind market replay...at that point in time, if you have run a lot of data through market replay, it takes too bloody long for NT to reset the replay101 account so you can start replaying again.

my question is: What is NT storing during Market Replay that slows down and sometime crashes NT?

[NinjaTrader_AdamP]

ATI user,

Thanks for your note.

The market replay data you download yourself through File > Utilities > Download Replay data or it was saved by having "Record for Market Replay" checked. This is tick data, so if you are downloading/saving a month/years worth of data it will most definitely be a large file, and may take some time to delete. It may be faster to manually delete it yourself in the Documents\NinjaTrader 7\db\data directory or set up some Windows scheduler that deletes this data every night when you are off your PC.

Please let me know if I may assist further.

[ATI user]

ATI user,

I am happy to assist you.

There is essentially two types of historical data. One is for market replay, and it is not accessed unless you connect to market replay. The only thing it takes up is hard drive space and you can clear it out at any time by deleting the appropriate folders in My Documents / NinjaTrader 7 / db / data The other is for charting and/or backtesting. This is located in your My Documents / NinjaTrader 7 / db / directory and you can delete the appropriate directories at any time.

You can disable saving historical information from charts and for market replay by going to Tools > Options > Data > Uncheck "Record for Market Replay" as well as "Save chart data as historical". However, there will always be some historical information in the NinjaTrader directory because its more efficient in plotting charts rather than having to download historical information every time you open a frequently used chart.

For more information on historical and real-time data, please see the following link : http://www.ninjatrader.com/support/h...rical_data.htm

Please let me know if I may assist further.

thanks...that however was not my question

I am asking regarding running Market Replay...please read original post above

I am referring to the 'history' that Market Replay stores when it is running that, when you run many days of data, takes NT forever to delete. This I assume is the same history that is in the 'Clear History' option (which is always default yes) in Simulator/Reset

[NinjaTrader_AdamP]

ATI user,

I am happy to assist you.

There is essentially two types of historical data. One is for market replay, and it is not accessed unless you connect to market replay. The only thing it takes up is hard drive space and you can clear it out at any time by deleting the appropriate folders in My Documents / NinjaTrader 7 / db / data The other is for charting and/or backtesting. This is located in your My Documents / NinjaTrader 7 / db / directory and you can delete the appropriate directories at any time.

You can disable saving historical information from charts and for market replay by going to Tools > Options > Data > Uncheck "Record for Market Replay" as well as "Save chart data as historical". However, there will always be some historical information in the NinjaTrader directory because its more efficient in plotting charts rather than having to download historical information every time you open a frequently used chart.

For more information on historical and real-time data, please see the following link : http://www.ninjatrader.com/support/h...rical_data.htm

Please let me know if I may assist further.

[ATI user]

Please explain exactly what the 'History' is that replay stores.

There is an option to "Clear History' when Replay101 (or sim101) is reset. What is that history?

Personally I have no interest at all in any history. I only want to process the data as if it was coming from live data feed. If there was an option to "Do Not Record History" I would always use it ALWAYS.

This question comes after replay running out of memory last night and my turning it off rather than dealing with it. This morning I could not connect to replay on 6 attempts...then realized NT was not frozen but rather clearing the replay account...of, I assume this 'history'....for 10 minutes. This is unacceptable and a complete waste of my time.

What is the history and why can it not be done away with?

[NinjaTrader_Brett]

If it has no if statement to filter out what it runs on. It will run on all series. A B and C. if you have a 3 series multi time frame/instrument script.

Let me know if I can be of further assistance.

[ATI user]

yes of course...I was referring to the code outside the if statements...it is executed on both primary and secondary series or just primary

[NinjaTrader_Brett]

You would just need to look at the if statements

if (BarsInProgress == 1)
//This code only runs in Bars In Progress 1.

if( BarsInprogress == 0)
//This code only runs in bars in progress 0.

[ATI user]

sure, but then it will run on all Bars. Primary Secondary and any others.

ok...so the old strat executed the `code A` for both the primary AND secondary on each BarUpdate...while the new strat executes primary OR secondary...no code being executed on BOTH series ...correct?

[NinjaTrader_Brett]

sure, but then it will run on all Bars. Primary Secondary and any others.