Daily Data False when Merge Policy set to MergeBackAdjusted

[Harry]

Dierk, I have renamed the folder <my documents>/NinjaTrader7/db to <*/*/dbold>. NT7 created a new folder during installation. Also there was no data, when I started NT 7.0.0.7 for the first time, and I reloaded everything from scrath via the Historical Data Manager, after I had selected the instruments (only the latest contract ES 03-10 was on the default list).

I also noted that the problem is mostly solved. But if you look at the rollover date, you will find false and missing data on rollover date and the the day after. Again, please have a look at the ES 03-10 chart, merged backadjusted with template 24/7 (screenshot) for December 10.

You needed to clear out all data including the cache by manually deleting all (!) subfolders in <my documents>/NinjaTrader7/db.
__________________
Dierk, NinjaTrader Customer Service
NinjaTrader is a FREE application for advanced charting, market analytics, system development and trade simulation.

View schedule of upcoming online product training events.

.

[NinjaTrader_Dierk]

I could reproduce the "island" issue. We'll into and will get back to you. Thanks for reporting.

[Harry]

It only seems to be a problem with the display of data around the rollover date. The data << rollover date is correctly adjusted and data >> rollover date is unadjusted as supposed.

So it is likely a local problem related to the session manager.

Please also remind the missing day for the old contract as shown below.

[Harry]

MergeBackadjusted Instrument Settings

I very much appreciate that with the setting <MergeBackadjusted> daily and intraday charts load the relevant data from all the futures contracts that cover the lookback period of the chart. However, I think it is important, to leave the choice for rollover date and offset with the user. There are two reasons for this:

(1) False or inappropriate default settings

(2) Practical need: It might well be that I enter into a position in the new contract a couple of days before rollover date. In this case I would modify the settings to prepone the rollover as to extend my contract data with the prior contracts.

Here is one example. The liquidity in Gold futures shifts directly from the August to the December contract. The NT 7 default settings, however, switch from August to Octobre, which is a non-liquid contract. The daily chart below easily confirms that the wrong contract was used.

Could you please rectify the default settings for GC and then think about an option for us to customize rollover settings?

[NinjaTrader_Dierk]

Thanks for your suggestion. We'll add it to the list of future considerations.

[Harry]

Contracted backadjusted to itself?

Observed today on rollover date for CL. NT 7.0.0.9 / TWS 900.6

When changing the lookback period of the chart, I observed

CL 03-10 (30 min, lookback 5 days) : correct
CL 03-10 (30 min, lookback 65 days): contract shows an offset at rollover date

So with the increased lookback period of the chart, the contract is backadjusted to itself, which of course is incorrect. The observed offset of 0.37 is the offset that should be used, when backadjusting CL 03-10 to Cl 04-10.

Any news with regard to the other problems for backajdusting?

[Harry]

Problem occurs with longer lookback periods only!

Just to be clear: I am not talking about the new contract, but the old contract, and how it is transformed. On rollover date you might find yourself trading the old contract, so you expect that it is displayed unchanged.

However, where nothing should be changed, NT creates some distortions. Below example of ES around rollover date:

Chart for ES 12-09 with a lookback period of 100 days (last day of chart period Decembre 18): False backadjustement before rollover date, nearly 1 day of data missing on rollover date, correct after rollover date only.

Chart for ES 12-09 with a lookback period of 15 days (last day of chart period Decembre 18): No backadjustment prior to Dec 11 (correct), data missing on rollover day, correct afterwards.

Findings confirmed:

- Depending on lookback period of chart, NinjaTrader introduces a gap within the old contract on rollover day. The artificial gap has the size of the offset to be applied, when old contract is joined to new contract.

- Problem does not only occur on rollover day! Also observed with December contract for ES.

[NinjaTrader_JoshP]

Harry,

Will get back to you at a later point in time.

[Harry]

Please make sure that development tests the MergeBackAdjusted issue for ES and Globex session template (GMT -6, Sunday 5:00 PM to Monday 3:15 PM, other days 3:30 PM to 3:15 PM). Test should cover at least 3 contracts (100 days). If ES/Globex works correctly on a chart with a lookback period of > 100 days, everything else probably will as well.

Harry,

Will get back to you at a later point in time.

[NinjaTrader_JoshP]

Harry,

I believe that would be the relatively the same scenario as the CL issue you reported, no? Please correct me if I am wrong.

[Harry]

There are two problems

(a) false backadjustment with longer loookback period
(b) problem that affects data at rollover date (but not before and after) that is related to sessions starting before midnight

(a) for sure can be observed with CL
(b) can be observed for ES/Globex, and I assume (but do not know) that it could be observed with CL (ETH), but certainly not with CL (RTH)

The point is that you need to test with a session template, where the first bar of today's session (= session end) has a date of yesterday. So you cannot test the problem with any RTH session.

As there is a number of other bugs related to ETH sessions (see pivots) - because these usually start before midnight local time, I definitely think it is necessary to test for these!

Harry,

I believe that would be the relatively the same scenario as the CL issue you reported, no? Please correct me if I am wrong.

[NinjaTrader_JoshP]

If I understood you correctly, there have already been changes for the next version around the area of rollover dates for (b) which should make it work better. Please try again then. Thank you.

[Harry]

When will 7.0.0.10 be out?

If I understood you correctly, there have already been changes for the next version around the area of rollover dates for (b) which should make it work better. Please try again then. Thank you.

[NinjaTrader_JoshP]

Harry,

It should be out here fairly shortly. Thank you for your patience.

[Harry]

MergeBackAdjusted works

NT 7.0.0.10

Just tested MergeBackAdjusted with 3 contracts of ES. Checked the data back until September. The first finding is that all data is correctly merged!

I will check this for other instruments as well.