Historical Data overwritten

[Beauregard]

Hello

I have for several years historical data, which I verified and do not want to touch. Data goes from 1998 until end of 2011 for example. When I run a strategy, being connnected to a provider, then my historical data is overwritten.
Same happens when I download with the historical data manager.
How can I avoid this? If I run a strategy (in strategy tab or analyzer) I do not want that existing data is overwritten.

thanks for your feedback

[Crassius]

Historical data overwriting is an annoying behavior of NinjaTrader that I and others have complained about in the past. The current implementation is lacking, but with a thorough understanding of how NinjaTrader behaves, you can overcome this problem.... its just not very user friendly... and you need to be careful you make no mistakes.

Go into the help section and thoroughly study the sections I have drawn arrows to in the attached screenshot. Especially the sub-section under Historical & Real-Time Data titled "When Does NinjaTrader Load Historical Data".

NinjaTrader doesn't give enough controls to allow the user the shape the way NInjaTrader will download and overwrite historical data. You need to completely understand and know the rules that are outlined in the highlighted Help sections and use the rules as your controls.

NinjaTrader should add more intuitive flexibility to historical backfill. One would have to think through the interface, but off the top of my head, a checkbox to prevent overwriting would be good. Better yet, MultiCharts .Net version has a control that allows you to select "Fill In The Gaps Only".... which leaves existing data alone, and fills in what you need. Multicharts got it right.... Ninja... not so much.

When I connect to my data provider with Ninja, I feel like I have to protect myself from NinjaTrader overwriting existing data. I carefully check under Options and consider what charts etc I have open that would cause an overwrite, on all workspaces, to make sure it behaves the way I need it to.

I have since moved my large database of historical data to a second dedicated development machine. This machine never connects to the data provider. I update it from files manually. This eliminates the potential overwriting mistake, but of course wasn't the only reason I moved development to a second machine.

Before I moved to a second machine, I created a fake symbol in instrument manager, and my multi-years worth of historical data for backtesting resided under that fake symbol, which protected it overwriting. I would then update it manually.

Even knowing all the rules, it is easy to make a mistake... leave a chart open on another workspace by accident... and you data can be overwritten by mistake.
Maintaining the years worth of backtesting data under a fake symbol protects it. I would then load a few months worth under the real symbol to work with live, and try hard not to overwrite that data by considering before each connection event, exactly how Ninja would behave.

You can make it work, but the first rule in good design is, Never Make The User Have To Think.

[Beauregard]

Hi
Thanks for your detailed information. I will read the help sections again carefully.

Indeed, I am also using fake symbols and a developement-test machine only, exactly for these reasons.
Still, the problem occurs with real trading, where I need also historical data and it must be reliable.
During my research with different data provider I found out, that especially some older historical data has very poor quality (including kinetick).

NinjaTrader Support: What are your plans about historical data handling?

thanks

[Crassius]

Hi
.....where I need also historical data and it must be reliable.
During my research with different data provider I found out, that especially some older historical data has very poor quality (including kinetick).

Exactly.... if you have spent the time to clean up the historical data, you don't want it corrupted by overwriting it.

Ninja really has this bit badly designed.

[NinjaTrader_AdamP]

Hello,

I took the liberty of adding a suggestion to our tracking system under ID #1989 for a feature request that would be better for your uses as outlined in your posts, specifically more options for handling historical data overwriting, etc.

[Crassius]

Hello,

I took the liberty of adding a suggestion to our tracking system under ID #1989 for a feature request that would be better for your uses as outlined in your posts, specifically more options for handling historical data overwriting, etc.

Thanks Adam --

Another example of NinjaTrader' excellent customer service. Nothing can improve your product better than listening to and responding to your customer's real world everyday experience with the software.

[Beauregard]

Thanks, this will be very helpful