Charting the wrong data for a future expiration?

[ntdev]

Hello. This one is weird.

#1. NT8 (8.0.23.2) is graphing the wrong data for the historical part of RB 12-21.

Repro#1:

You will notice that I have added 3 instruments into the Market Analyzer. RB 12-21, RB 05-21 and RB 04-21.
You can see that I created a chart with RB 12-21 (Line on Close)
You can also see that I have the correct values in the Historical Data -- Closes in the 1.7xxx price range.

Expected: Chart should represent the data.

Actual: (see below)

The historical data plots as 2.0xx and once it starts showing real-time data that will be plotted with the correct value of around 1.7xx.



Repro #2: (with an indicator)

I call to add 3 data series to my chart which has a primary instrument of CL 04-21.

AddDataSeries(RB 04-21, Minute, 1)
AddDataSeries(RB 05-21, Minute, 1)
AddDataSeries(RB 12-21, Minute, 1)

I then have some debug code in OnBarUpdate which shows the output above.

For some reason it is printing the same values for all 3 indicators, UNTIL it gets to real-time data and then it shows correctly but not all instruments are updated for every minute.


Can you reproduce on your machine? At least just the chart.

Any ideas? It's like NT is getting confused on what Last[0] means but I also tried to get the value with BarsArray[BarsInProgress].GetClose(BarsArray[BarsInProgress].CurrentBar) and that also returns the same incorrect value.

But it's even more strange that the incorrect value is being plotted incorrectly in just a simple chart.

[ntdev]

What might also be weird is that I tried to load a month of data but nothing shows up.

And data doesn't load for RB 05-21 either but it does load for RB 04-21.

Yes, I tried removing all data first.

It is as if it doesn't have the data for the December month so it uses the April data in it's place.

My data provider is Continuum.

[ntdev]

PS It clearly only adds the data that it gets from the real-time feed for the future months.

Here's what my data looks like after deleting RB 12-21 completely then letting it sit for 5 minutes.



So the bug could be, "Data Not Downloading and No Error Message."

[NinjaTrader_PaulH]

Hello ntdev,

Thanks for your post.

In short, what you see on the chart for a future contract (not the current contract) is expected with the default historical data merge policy.

NinjaTrader by default will merge historical data based on the current contract (RB 04-21 in this case). That is what you see on the RB 12-21 is the historical data from 04-21 and then as you observed the real time price of the RB 12-21 contract is quite different. There is not that much difference pricing-wise between the 04 and 05 contracts so it is not as observable. In all cases the real-time data will be correct, it is just a matter of what historical data you need to access.

Please review the historical data merge policies here and decide if you want to go through the process to change the merge policy of the historical data: https://ninjatrader.com/support/help...rge_policy.htm

https://ninjatrader.com/support/helpGuides/nt8/?merge_policy.htm

The policy is set through Tools>Options>Marketdata https://ninjatrader.com/support/help...marketdata.htm

[ntdev]

Hey Paul.

Thanks for the quick response.

#1. Data Fix for RB 12-21 << I want to chart the differences between different expiration contracts so I need this data.

I set the Global merge policy as "Do not merge".
I deleted all the RB *.* data files.
I tried to re-download the data for RB 12-21 -- * BUG: no data is downloaded *
I tried to re-download the data for RB 04-21 and RB 05-21 -- this worked

The data exists for RB 12-21 and it does trade -- Q1: how to get this data?

Here is is the debug output, but there is no data on my drive.

2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/10/2021 12:00:00 AM' period='Daily'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/10/2021 12:00:00 AM' period='Daily - Ask'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/9/2021 11:59:59 PM' period='1 Tick - Bid'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/9/2021 11:59:59 PM' period='1 Tick'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/9/2021 12:00:00 AM' period='1 Minute'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/9/2021 11:59:59 PM' period='1 Tick - Ask'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/10/2021 12:00:00 AM' period='Daily - Bid'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/9/2021 12:00:00 AM' period='1 Minute - Ask'
2021-03-09 15:31:35:732 Cbi.Instrument.RequestBars (to Provider): instrument='RB 12-21' from='1/1/2021 12:00:00 AM' to='3/9/2021 12:00:00 AM' period='1 Minute - Bid'

Q2: What exact settings would you recommend for my case?

#2. For the main instruments that I trade, merging is a nice to have feature and useful so that I can compare with the previous month.

Q3: Is it possible to set the merge policy on just for these 2 instruments where I need to see the actual data without merging but leave everything else to merge?

Q4: If I am not able to have this merging feature turned on then can you tell me exactly what are the steps that I would need to do in order to manage this manually? And what do I lose if I need to turn it off?

Q5. Can I do the data request to download this data in my code?


It seems very reasonable that you should be able to get accurate quotes for any instrument that you may trade. Trading multiple contract months or hedging one from another seems like a normal thing that should be supported. Not sure how to write a feature request yet if this is not possible. It depends on how you answer my other questions.

Thanks in advance.

[NinjaTrader_PaulH]

Hello,

Thanks for your reply.

What data feed are you connected to?

[ntdev]

What data feed are you connected to?

Ninja Trader Continuum

For Q1... I do have an exotic workaround which involves the fact that I wrote code to read and write my own .ncd files along with a daemon / scraper that allows me to use data from TradingView.com inside NinjaTrader. RB 12-21 is available on TradingView.com as RBZ2021 but it's 10-minutes delayed with my current plan, so I can backfill missing historical data then let it go live.

But the fact that I can get the data in real-time on NT but not download it historically for some future indicators and not others definitely seems to be a bug.

Q2: What exact settings would you recommend for my case?
Q3: Is it possible to set the merge policy on just for these 2 instruments where I need to see the actual data without merging but leave everything else to merge?
Q4: If I am not able to have this merging feature turned on then can you tell me exactly what are the steps that I would need to do in order to manage this manually? And what do I lose if I need to turn it off?
Q5. Can I do the data request to download this data in my code?

[NinjaTrader_PaulH]

Hello ntdev,

Thanks for your reply.

Yes, you can manually set the merge policy per master instrument (not per contract). Please see the help guide here: https://ninjatrader.com/support/help...GeneralSection

No, you cannot set the merge policy through Ninjascript, however, you can see what the policy is per instrument: https://ninjatrader.com/support/help...ergepolicy.htm