data differs from source

[NutFlush]

Hi,

I have a data problem:

The datafeed I am using is from IB. When I look at the IB chart for XIV on the 1-minute scale for today at 10:01 (exchange time) I see the HIGH at 24.00$, but in my NT chart the HIGH is only 23.99$

I double-checked with my TradeStation application and there were several ticks at 24.00$ during that minute.

The strategy that runs on that data also works with the HIGH of 23.99$, so it's not just a display problem, but rather a data problem.

Any idea what might be causing this?

Thanks,
NutFlush

[NinjaTrader_PatrickH]

Hello NutFlush,

Thank you for your post.

So I may investigate this matter further please answer the following questions:

What version of NinjaTrader are you using? You can check under Help -> About (Example: 7.0.1000.X)

What version of TWS are you using? You can check under TWS -> Help -> About -> Scroll to the Build #

Can you provide a screenshot of both the data in NinjaTrader and TWS? If so, then please attach the screenshot to your response.

I look forward to your response.

[NutFlush]

Hello NutFlush,

Thank you for your post.

So I may investigate this matter further please answer the following questions:

What version of NinjaTrader are you using? You can check under Help -> About (Example: 7.0.1000.X)

What version of TWS are you using? You can check under TWS -> Help -> About -> Scroll to the Build #

Can you provide a screenshot of both the data in NinjaTrader and TWS? If so, then please attach the screenshot to your response.

I look forward to your response.

Hi,

I am using: NT 7.0.1000.14 & TWS 4.0 (938.1e).

Sorry, but I cannot give you a screenshot as I restarted NT - and now the bar is shown correctly.

I suppose the issue is in how the bar is built. NT collects tick data from IB and builds the bar with that info, right? So if it misses ticks then the bar may look different.

Why could the software have missed ticks? Is that a problem with IB's datafeed or rather a NT issue?

Cheers,
NutFlush

[NinjaTrader_PatrickH]

Hello NutFlush,

Thank you for your response.

Interactive Brokers' real-time tick data is aggregated, therefore you can see a difference after you reload historical data from the Interactive Brokers servers as the historical data is not aggregated data.

Please let me know if I may be of further assistance.

[NutFlush]

Hello NutFlush,
Interactive Brokers' real-time tick data is aggregated, therefore you can see a difference after you reload historical data from the Interactive Brokers servers as the historical data is not aggregated data.

Hi PatrickH,
can you please explain what "aggregated" means?

If several ticks happen at the same price then I can understand that an aggregation can work by simply summing up the volume of all ticks and report that as a "single tick".

- But what happens if the ticks trade at different prices and those get "aggregated"? Does IB report those ticks at a single price?
- Does IB aggregate ticks that belong into 2 consecutive candles (i.e., two 1-minute bars) so that the volume and price info only goes into 1 candle instead of being split between 2?

Is there any additional info about that issue that you could direct me to (i.e., links)?

Thanks,
NutFlush

[NinjaTrader_PatrickH]

Hello NutFlush,

Thank you for your response.

Aggregated refers to the data being a snap shot and not tick by tick data, however after you reload all historical data you will see the corrected data. This means there can be several ticks in one tick bar. I do not have any links to provide on this item. If you perform a search on Interactive Brokers data you will find several articles on this matter.

[koganam]

Hi PatrickH,
can you please explain what "aggregated" means?

If several ticks happen at the same price then I can understand that an aggregation can work by simply summing up the volume of all ticks and report that as a "single tick".

- But what happens if the ticks trade at different prices and those get "aggregated"? Does IB report those ticks at a single price?
- Does IB aggregate ticks that belong into 2 consecutive candles (i.e., two 1-minute bars) so that the volume and price info only goes into 1 candle instead of being split between 2?

Is there any additional info about that issue that you could direct me to (i.e., links)?

Thanks,
NutFlush

That is just in the nature of aggregated feeds. What comes out depends entirely on the aggregation protocol, and is affected by timing, latency, granularity, among other factors. I do not believe that any aggregated datafeed provider has ever published their method/protocol.

The only real choice if your strategy is so sensitive to data precision is to get a raw, unfiltered, unaggregated feed. Any and all data massage before delivery to the end point, is a cause for potential variance. As soon as the data is massaged, it is ipso facto impure! Do not let your provider contaminate your feed: if need be, get a different provider. You can always use pure data to execute trades on any other broker.

[ajcpereira]

Hi Koganam,

Do you have any suggestions of tick data providers as you look so experienced?

Thanks for your help.

[koganam]

Hi Koganam,

Do you have any suggestions of tick data providers as you look so experienced?

Thanks for your help.

I do know of others, but as an NT partner, the only datafeed that I now recommend is Kinetick. Look at my signature for some links.

That having been said, exercise your Google-Fu on "unfiltered data feed stocks futures", and you should get many others. NT also comes with official datafeed partners. You can explore those from the NT home pages.

[Harry]

Did you use Renko charts? Renko charts never show the exact high or low.

Hi,

I have a data problem:

The datafeed I am using is from IB. When I look at the IB chart for XIV on the 1-minute scale for today at 10:01 (exchange time) I see the HIGH at 24.00$, but in my NT chart the HIGH is only 23.99$

I double-checked with my TradeStation application and there were several ticks at 24.00$ during that minute.

The strategy that runs on that data also works with the HIGH of 23.99$, so it's not just a display problem, but rather a data problem.

Any idea what might be causing this?

Thanks,
NutFlush

[Harry]

I have opened a NinjaTrader chart for XIV with IB as historical data feed. NinjaTrader plots the high exactly as $ 24.00, see chart attached.

That leaves us with three possible explanations.

(1) You used real-time data which was inaccurate. This is well possible as Interactive Brokers data is snapshot data that does not include all real-time ticks.

(2) Your system clock was a few seconds off. IB's real-time data comes without time stamps and NinjaTrader uses the system clock to timestamp real-time data.

(3) You used Renko bars, as assumed below.

When NinjaTrader is fed with correct historical data, it displays the 10:01 bar's high correctly as $ 24.00.

In any case the IB data feed is not a quality data feed. You get what you pay for. It is best to use Interactive Brokers together with Kinetick or IQ feed. First-connect to Kinetick for the datafeed, and second-connect to InteractiveBrokers for order execution, and you are fine.

[NutFlush]

Hi Harry,

first, thanks for taking the time to deal with my problem!

I used the regular 1-minute bar chart (no Renko). In the meantime I had chats with 2 reps from IB and they confirmed that their data is not tick-by-tick data, but rather what they call "snapshot" data. Unfortunately they don't say anything about that on their website! Of course they don't claim to give you tick data, but what other data feed would make sense??? They couldn't even explain to me or point me to a document that explains what "snapshot" means (other than saying there are 4 snapshots per second).

It appears that they pass on the price at the moment of the snapshot, but not the price RANGE since the last snapshot. That would explain why the HIGH of a bar in my chart (based on live data) was below the historical bar (when using historical data) because the tick(s) that happend at the HIGH were not captured by any of the snapshots.

I have come to the same conclusion as what you suggested: I need a datafeed for tick data and will consider Kinetick and IQFeed. Currently I lean more towards IQ, but I would be grateful for any further info or a comparison of data providers.

Cheers,
NutFlush

[Harry]

You have well explained the shortcomings of IBs snapshot data. The real-time volume data is even worse. However, what you can do, is regular refresh the chart via F5. This will replace real-time data with historical data, which is accurate. Interactive Brokers offers historical minute data, but not historical tick data, so you would not want to refresh tick charts.

When comparing IQ feed and Kinetick, they are pretty similar. Kinetick is a rebranded IQ feed that lacks a few instruments and options. For example Kinetick does not give access to ICE/Nybot/IPE. Otherwise I have made good experiences with Kinetick.

If you use other trading software besides NinjaTrader, or if you want to trade Russell 2000 or Brent Crude futures, you would go for DTN/IQ, otherwise Kinetick is sufficient.

[NutFlush]

Apart from the initial fee it appears to me that IQFeed might actually be cheaper than Kinetick for my purposes (US stocks & ETFs, Emini). Do you know about any disadvantages of IQFeed (especially compared to Kinetick)?

Kinetick is advertised as being especially well integrated into NT. Is IQFeed worse in that respect for example? How about their customer support?

In the TradeStation platform I can set that any ticks outside the NBBO (National Best Bid Offer) can be ignored (this avoids "spikes" or "bad ticks" that might be far away from the current price). Is that something I can set in NT or must/can the data provider already filter out those ticks (if so: can IQFeed & Kinetick do that)?

Cheers,
NutFlush

[NinjaTrader_PatrickH]

Hello NutFlush,

Thank you for your post.

You can set Filter bad ticks % off market in NinjaTrader through Tools > Options > Data.

Please let me know if I may be of further assistance.