Ohlc

[ScottB]

Hi,

Is there any reason why I am getting different OHLC values in real time for a bar then when I reload the data? The 6E 12:23 bar today (10/04/2010) showed one set of values when the bar plotted in real time that didn't match another data feed (TradeStation).

Then when I reloaded all historical data, they matched. That is not a particularly good thing in my opinion but I am holding out that I don't understand how something in this process works.

OHLC
TradeStation
1.3687/1.3688/1.3686/1.3688

Ninja in real time
1.3687/1.3689/1.3687/1.3687

Ninja after reload historical data
1.3687/1.3688/1.3686/1.3688

I have tried this on several bars now and it is definitely an issue - yikes. Please tell me this is something I am doing wrong (lie if you have to).

I just verified the same thing is happening with the volume; one value when plotted in real time and another after reloading. This actually seems a bit more reasonable than the price issue depending on which bar you assign a given tick to.

Scott

[NinjaTrader_Kyle]

Hello ScottB,

Thank you for your post.

Which data provider are you using with NinjaTrader?

[tulanch]

thought I saw same thing with volume bars...

wow, I thought I was seeing things this past Sunday.......strange to see a post about a similar issue I faced

This past weekend I uploaded 6.5.18 and after this update my rangebar chart was not printing the same high of day as I had on a 5 min chart for the same ES12-10 - as I only have this single instrument loaded up in NT.

So I then shut down my NT connection, did a database repair, shut down NT completely, then shutdown my computer and did a reboot....after I got back on and re-connected live all was better...

it has been working since....

[ScottB]

Hi,

Kyle, my data provider is Trading Technologies. I have now verified this on many different bars although all are on the 6E (December contract).

Scott

[ScottB]

tulanch, I am on NinjaTrader 7 build 22; have you any experience with that release doing the same thing?

I will give your idea a try. Geeze this is a production system and I must say, this is one of the scariest situations I have run across trading algorithmically.

Scott

[NinjaTrader_Kyle]

Scott,

Trading Technologies themselves do not provide historical data. This data is provided by NinjaTrader as a courtesy service. The data itself was recorded using another feed, Zen-Fire.

Since different data is used, you may see OHLC values change when data is reloaded.

From our Help Guide:
"Different data feeds produce different charts (especially when using tick based intervals vs time based intervals). Market data vendors each employ various methods for tick filtering, throttling and time stamping. As a result, no data stream is 100% identical and thus can cause subtle differences in charts. Since NinjaTrader supports many of the leading brokerage and data feed technologies, its guaranteed that two traders using NinjaTrader on different data feeds will have minor differences when plotting the same market and time interval.

Moving forward, the only way to ensure that real-time and historical bars match would be to use a data provider who offers historical data themselves.

[ScottB]

Kyle, maybe I wasn't clear; the historical data is correct, the live data is not. I am putting in a call to my broker to see if there are known problems with the TT datafeed.

Scott

[NinjaTrader_Kyle]

ScottB,

I know of no issues with the TT data feed, but just to be clear, that OHLC values differ between providers is not indicative of an issue.

[ScottB]

Kyle,

A given trade can fall on a boundary and one data provider puts it in one bar and another one puts it in a different one. With a liquid contract like the 6E, those differences can easily account for a 1 tick difference but if it is more than that, there is a problem in my opinion.

I didn't mean to imply it is a NinjaTrader problem but it is still very scary to get different prices depending on whether it is live or historical.

Under any circumstances, repairing the database seems to have done the trick.

Scott

[ScottB]

I have verified that all is still working so repairing the database does the trick. I do have one observation/question.

Given that CME datafeeds are timestamped at the millisecond level, how can a tick be put in the wrong bar, especially during the middle of the day in a very liquid contract i.e. no missing bars, etc?

Are there any known issues that could indicate I need to repair the database (beyond the obvious, the data isn't right) as I don't usually check the data as carefully as I was when I discovered this and I would like to avoid a repeat of the situation.

Thanks in advance
Scott

[NinjaTrader_Kyle]

Hi Scott,

Can you elaborate on that first question regarding CME time stamping? (Your asking in terms of real-time or historical data?)

[ScottB]

Kyle, I have worked with the raw CME data feeds and I know they are timestamped at the millisecond level. I was wondering how a trade could get into the wrong bar on a time based chart.

For tick charts, etc. it is clear that the bar depends on when you start counting and so I would expect there to be differences among tick bars from different data feeds or even from the same data feed live, vs historical.

More musing than question. I am trying to put some of our relatively high frequency trading applications into NinjaTrader because it is infinitely easier to make adjustments with NinjaTrader vs. a pure FIX application written in C++ or C#.

These algorithms are very sensitive to the data and I need to be able to sync up all our various data feeds for testing purposes and they all agreed except for the anomoly I mentioned at the beginning.

However, my earlier post may have sounded, I love NinjaTrader and you guys are the best support staff I have ever dealt with (including some of my own guys).

Scott

[NinjaTrader_Kyle]

Hi Scott,

Thanks for the clarification and for the kind words!

With Trading Technologies, the real-time data is actually timestamped by your local PC's clock - TT does not send the timestamps on with the data.

When historical data is reloaded from a server, that data is timestamped. So, if your PC's clock (used to timestamp in real-time) and the historical data servers clock are out of sync, even by fractions of a second, you could see instances where a bar's OHLC values change when historical data is reloaded.

The impact of this can be minimized by syncing the PC's clock.
You can sync your PC clock by double clicking on the clock in the lower right corner of your desktop. Once you have done that, click on Internet Time tab and then click Update. Your PC clock should now be updated.




Beyond this, you happen to be loading historical data that was recorded using a different data provider all together. So, you may also see instances where bar OHLC values are not exactly the same when data is reloaded, even when the PC's clock is perfectly in sync.

For your type of system, it may be wise to migrate to a data service who provides their own historical data and also offers native time stamping on their real-time data. If you're interested in a list of providers who meet these criteria, please let me know and I'll provide one.

[ScottB]

Kyle, thanks for the update, I never realized TT didn't timestamp the data (the TT data feed I used must have been timestamped by the firm's servers and they were synced with an atomic clock that is accurate to something like 19 microseconds).

I normally sync my clock several times a day but you are right that my clock is likely slightly out of sync, I haven't done it today.

If you have a list of those data providers, I would appreciate it; what is one more data feed in here

Thanks again,
Scott

[NinjaTrader_Kyle]

Hi Scott,

The following providers offer futures data, natively timestamp that data and also provide their own historical data:

• BarChart
• eSignal
• IQ Feed
• Kinetick (NT 7 only)
• PFGBest.com

Barchart.com (subscriptions starting at $45/monthly):

https://www.barchart.com/register/realtime/?ID=NJA

IQFeed offers a 7-day free trial for NinjaTrader users.

https://www.iqfeed.net/ninjatrader/index.cfm?displayaction=start&promo=1717496

eSignal promotional rate for NinjaTrader clients:

http://www.ninjatrader.com/webnew/eSignalSpecialNinja08.pdf

For more information about using NinjaTrader with multiple data providers:

http://www.ninjatrader-support.com/HelpGuideV6/MultipleConnections1.html