Loss of Data on Cloud Virtual Machine

[Turtle Trader]

Hello,

I am seeing a loss of ZenFire data points on 2 range and tick charts on a rented Virtual Machine at Gigenet Cloud in Chicago.

When compared to my local office machine, Ninja Trader on the VM sometimes has missing bars. Both machines are on a live Zen data feed. When I tell Ninja on the VM to reload the data it fills in the missing data and then looks like the chart on the local machine. The local machine is connected to the internet via a DSL line.

Here is what tech support said at Gigenet:

"Does the NinjaTrader software happen to be using UDP as the layer 4 transport? if so, this may be an issue with the software as re-transimission of packets must be handled by the programmer when dealing with connectionless protocols."

Please advise.

[NinjaTrader_Brett]

Hello,

This would be more of a question you would want to fire off to zen-fire.com and they will be able to let you know on this.

However what I can tell you is this:

"Does the NinjaTrader software happen to be using UDP as the layer 4 transport? if so, this may be an issue with the software as re-transimission of packets must be handled by the programmer when dealing with connectionless protocols."


Even if this is the case, the Zen-fire server never retransmitts packets even in the above case. if you miss the packets you miss the packets. Zenfire is either you recieve the unfiltered tick data or you dont. If you have a momentairy drop in connection and loss a few packets they are lost.

This is done for maximum speed.

Reloading the historical data is the only way to bring that back in.

This is the reason why this occurs. Please see this guide on why my chart looks different after reloading:

http://www.ninjatrader.com/support/helpGuides/nt7/how_bars_are_built.htm

Understanding why a chart can look different after reloading historical data from the server

Let me know if I can be of further assistance.

[Turtle Trader]

reply -- found the problem

I found one aspect of the problem, this should be easy to recreate as I've seen it now on two different machines.

I have a reference machine for charting several instruments and a trading machine that has only a couple of charts open. The reference machine is also running a market analyzer for each instrument I'm watching. This is about 10 Futures instruments.

On the trading machine I may only have 2-3 charts open and there is normally no other analyzer or strategy running.

The problem is when I change the instrument on the trading machine chart it often seems to skip some of the data when it loads the new instrument on an existing new chart. I can tell because bar data is missing that shows up on the reference machine.

If I have a market analyzer open on the trading machine in the background with a constant data stream for that the instrument, then the data appears to be charted correctly when I change instruments. I suppose then it gets the bar data from a cache?

If I don't have the market analyzer open on the trading machine, I can force the trading machine chart to synch with the reference machine by forcing it to reload the historical bars.

There may be other aspects to this data loss.

[NinjaTrader_Brett]

Hello,

Thanks for the note. This most likely is expected what your running into.

Most likely the chart is not loading historical data since it does not meet one of the following rules to cause a relaod of historical data from the server. Therefor you end up having to do a manual reload.

http://www.ninjatrader.com/support/helpGuides/nt7/historical_data.htm

When does NinjaTrader load historical data?

Let me know if I can be of further assistance.

[Turtle Trader]

reply

I was expecting that when I changed the instrument on the chart that it would reload the historical data for the new instrument...looks like that was an incorrect assumption.

thanks.

[Turtle Trader]

clarification

Hello,

Can you clarify this documentation: (from the link you sent below) : "The Market Analyzer option is not as viable since it only maintains a 100 bar cache as a default setting that can be changed. Since 99% of all charting requests consist of much more than 100 bars, they will most likely incur a data load."

How do I set the cache...is that the "# bars to look back" on the market analyzer / properties dialog?

Thanks

[NinjaTrader_Brett]

Hello,

Yes this is #Bars Look Back setting that is discussed there.

Let me know if I can be of further assistance.

[Turtle Trader]

reply

Thank you. One final question:

When Market Analyzer first loads, does it load historical bars to fill the cache, or does it fill them up as they come in real time?

[NinjaTrader_Brett]

Hello,

It will load to fill it in. Based on the same data load rules as charts.

Let me know if I can be of further assistance.