Edit historical data programmatically

[vtaran]

Historical forex data loaded from IB has many bad bars. In NT6.5 they are filtered, but in NT7 they are not. Correcting them in Historical Data Manager manually or by expotring-editing-importing will take a lot of time. I would like to correct them programmatically. Will you publish the structure of NT7 historical .ntd file?

[NinjaTrader_Dierk]

>> In NT6.5 they are filtered, but in NT7 they are not
Incorrect. Neither Nt6.5 nor NT7 would filter any bars in regards to "prices being off".

Unfortunately we don't plan to publish the data format for the NT internal data structures.

[vtaran]

Just load min bars from IB for $GBPUSD for the last 5 days and observe the difference. 13 bad bars in NT7 and none in NT6.5. It makes IB forex backtesting in NT7 almost useless without first correcting historical data.

[NinjaTrader_JoshP]

vtaran,

I just tested this on 6.5 and 7 and do not experience what you are seeing. "Bad" bars in 7 are also there in 6.5. NT just acts like a data receiver and whatever data is coming through from the data provider is what is charted.

The data on 6.5 looks to match 7's bar for bar as seen in my attachment.

[vtaran]

Josh,
In you sample (just over 2 hours) "bad" bar is reasonably normal. Check the bars over a week of data. It's easier to see it on 60 min bars. I reloaded data again. The same result - NT6.5 corrects "bad" bars and NT7 does not.
See my attachment.

[NinjaTrader_JoshP]

vtaran,

I have checked the full length of my chart and cannot reproduce. Suggest you try reloading your historical data cleanly with removal of all old data.