ZenFire lag

[Alfred]

Thanks VTtrader for the chart plot !! ...

I am curious what may be considered normal and also acceptable re lag.
Attached 2 typical current charts...

3000 Mhz, 1024 memory...1 workspace mainly (13 charts, no indicators except 2 lag studies)...Zen-fire data....only Ninja running...CPU 20 to 40% usually or so this morning...

The lag at the 7am Pacific report spike got up to 28 or so seconds.
The lag for the majority of ticks seem to be under 1 second though.

Thanks

[VTtrader]

Alfred,
Glad you find it useful. BTW, the choice of chart is optional, the charts I posted are 34tick charts. It will give an "average lag" for the bar if used on something larger than 1 tick.

As for "normal" or "acceptable", I just know what I've seen. I would eyeball the average of the indicator on a 34t chart of ES runs between .3 - .4, with spikes (some 10 or so bars long) that hit around 1.5 - 1.7. Except for the 10:00 news release which shot up to 16 or so...and that is averaged over 34 ticks!

I would say that since it is compared to your computer clock, the amount the clock is out of sync will effect the results. I synced mine this AM to .001 seconds, but right now it says .2 seconds off. So there is certainly some variation due to the PC clock. An additional source of lag could be how accurately the "syncing process" is. When I sync my clock with the app I use (not windows) it will usually take 3 adjustments or so before I get something as close as .001 seconds. Even if I sync my clock everyday, it will slip by .9 seconds easily.

My feeling is under .5 would be very good, .5 - 1 is probably OK, over that wouldn't make me happy if it was the "norm". And obviously at 16 seconds I wouldn't even attempt to trade, since you have no idea where the market actually is.

Hope this helps.
VT

[Alfred]

VTtrader...

Thanks for the info...

What do you use to sync your clock ??

Have been using Atomic Clock Sync...it auto syncs at start up each morning...but not sure how accurate it is or remains throughout the day.

update...switched Atomic Clock Sync to auto sync every 10 minutes...that should be good enough...

Thanks...

[tomgilb]

Zenfire lag

Well, the lag issue continues today. I was OK until the news announcement then zenfire couldn't keep up.

Yesterday I installed trader20's TickTime indicator,
(see http://www.ninjatrader-support2.com/vb/showthread.php?p=117702)
and it showed today that I was lagging 60 seconds and climbing after the news announcement.

(VTtrader's mod to the TickTime plots it on the chart!
See http://www.ninjatrader-support2.com/...tpost&p=117811 )

Prior to this lag, no overnight bars were drawn and Reload Historical Data was grayed out, and I had to restart Ninja to get the missing bars. See my other post on this:

http://www.ninjatrader-support2.com/vb/showthread.php?p=117884

I don't know whether I have a data-delivery problem or a charting-platform problem. Either way, I can't trade under these conditions. Pathetic.

Reliable accurate timely data and its proper handling is the final frontier of electronic trading.

[Alfred]

Attached a 1 tick chart...with TickTime (white window) and TickTime Plot...black chart...in the chart there are a number of ticks showing lagging above 1 second in the last several minutes....but it appears that nothing is showing above 1 second lag in the white window for that same time frame...

Watching the two update live recently a number of lagging ticks (redlines) on the chart showed...but concurrently nothing showed above 1 second in the white TickTime window...

Reviewing this....from 1 03 00 to the close at 1 14 59 there were no ticks lagging above 1 second (TickTime) in the white output window but there were a bunch showing lagged above .08 second in TickTime Plot indicator (black chart)...hope I am reading this correctly...??

Any ideas on this...??

Thanks...

[VTtrader]

Alfred,
I use NISTime-32 to sync my computer clock.
As for the difference, I'll look at it again and see. I've been using it on a 34tick chart, but don't think that would make a difference, I'll take a look and post back later.

I was observing this afternoon, and did notice a few things though...see the attached pix.
In the pix in my previous post at 10:00, my clock was only off by .2sec, so the 16sec lag was way more than just the clock.
In this pix, I noticed the "lag" creeping up to 3.4sec, I checked my clock and it was 2.9 seconds off, I adjusted it and you can see the drop to -3.2 in the lag. I checked the clock again and it was off 2.9 still (other direction I guess) and adjusted it again and the "lag" came back in line. This points out how crucial the clock sync is before thinking it's a data issue. Later in the afternoon there were 2 more examples where the indicator started creeping higher, but checking my clock, it was off .7 and .5, which accounted for the higher values. The spike at the close I had just adjusted the clock and it was only off by .019, far less than the 7 second lag.

What this shows is how important the clock sync is in using this (or original TickTime) to determine if your data is lagged. This can be a useful tool to show a possible data issue but you want to verify, that it is the data and not your clock.

My other computer's clock is very stable, so now I have to investigate why my trading computer's clock drifts so much.

VT

[Alfred]

VT...

Atomic Clock Sync can be set to any number of seconds to auto sync your clock throughout the day. Have mine set to auto sync every 300 seconds or 5 minutes now...

http://www.worldtimeserver.com/atomic-clock/

Perhaps it may be worthwhile to run TickTime and TickTime Plot (on a one tick chart) tomorrow and compare lag between them...

Thanks for your work in this area....

[VTtrader]

Alfred,
Thanks for the tip, I downloaded and tried ACS earlier, since the automation interval option seemed more flexible than NISTime-32, which will only check hourly. I still like how NSITime will check and tell you how far off the clock is, so I may still use it to check the clock if I see a lag.

I did a little research about why my clock might be losing time so frequently while my computer is on. I found on the MSFT website an article that suggested it might be due to Advanced Power Management being enabled in BIOS. I haven't had a chance to check it, but will on my next reboot.

http://support.microsoft.com/kb/189706

As for your issue regarding a difference between TickTime and TickTimePlot. I just did a test on ES 1 tick chart, the values in the output window and in the data box were identical for every bar (except some values were rounded in the data box to 2 decimals). Granted this test was done during the overnight session, but I don't see why that would matter. BTW, I was using the default 0 and false in TT, and .8 and both true and false in TTP. I will check again tomorrow during the day when I get a chance.

VT

[Alfred]

VT...

Think I have settings correct now and TT & TTP seem to match OK...

Thanks !! ...

[JS999]

What is the significance of CalculateOnBarClose = true for the tick time and tick time plot indicators? It seems that when I enable that, the time lag spikes very significantly... this is on a one tick chart.

[VTtrader]

JS999,
I'll take a stab at it. I looked at it after you posted and can see what you're talking about.
I know TTP needs to have CalculateOnBarClose = false to function properly since it is averaging each tick that occurs intrabar. In order to capture those ticks it needs to run the code on each tick.
However, on a 1 tick chart, you wouldn't think it would make a difference. Since I ran the test about 11:30 PM the trading is very thin. I think what is happening might be due to the fact that the bar actually "closes" on the first tick of the new bar, and since the ticks are less frequent overnight, you get big lags.
That's my best guess anyway. I'll try it in the morning and see how that goes. My suggestion is to run it with CalculateOnBarClose = false, since that is how it's designed to function. As far as system resources go, it shouldn't make a difference with COBC set to true or false on a 1 tick chart, since every tick will still get processed.

Hope this helps.
VT

[JS999]

JS999,
I'll take a stab at it. I looked at it after you posted and can see what you're talking about.
I know TTP needs to have CalculateOnBarClose = false to function properly since it is averaging each tick that occurs intrabar. In order to capture those ticks it needs to run the code on each tick.
However, on a 1 tick chart, you wouldn't think it would make a difference. Since I ran the test about 11:30 PM the trading is very thin. I think what is happening might be due to the fact that the bar actually "closes" on the first tick of the new bar, and since the ticks are less frequent overnight, you get big lags.
That's my best guess anyway. I'll try it in the morning and see how that goes. My suggestion is to run it with CalculateOnBarClose = false, since that is how it's designed to function. As far as system resources go, it shouldn't make a difference with COBC set to true or false on a 1 tick chart, since every tick will still get processed.

Hope this helps.
VT

Thanks VTrader... as of now in the middle of the night my lag is on average .5 seconds or below... which still isn't perfect as far as I'm concerned, but I can deal with it. Are you guys still seeing crazy lag during news releases, and is this something recent with Zen-Fire? This is completely unacceptable if that is the case, since it breaks the very first rule of providing real-time data - namely, it needs to be in REAL TIME. Not 15 seconds behind. I will check it out tomorrow, but are you guys still seeing this on a regular basis, and is it also in heavy trends or only on news releases? Ideally, I would like to see some confirmation from Ninja people here on whether or not they are seeing this from Zen-Fire, and if using NT 7 is making any difference.

Thanks!

[JS999]

Well, I'm happy to say that I tested it out this morning with the release of the unemployment number and I get no significant lag on either crude oil or ES. The lag is pretty constant between 0 and .7 seconds, which could be a bit better, but is definitely tradeable.

As a side note, I have multiple virtual machines set up, each with their own connection to the zen-fire servers. While they are all going through my single internet connection, each of them is a separate ZF connection in its own right... so I would expect that if there was significant lag that they would show some discrepancies in terms of what they are displaying. As it turns out, they are all pretty much synchronized... this also tells me that there isn't much lag, even if I didn't have the tick indicator to tell me that. If anyone can see a flaw in my reasoning please point it out... would multiple separate connections show the exact same lag if they were all running from the same machine, or would each have different results? Obviously if the lag is related to the internet alone, then I think they would show the same prices... but if there was a problem with the ZF servers themselves, then they should all be somewhat off from each other, correct?

[NinjaTrader_Ray]

Hi All,

I have been in touch with Zen-Fire and here is what we know:

- The problem has been identified
- It does not affect everyone but small percentage of people
- It only occurs to people connecting through the internet
- They are working to address this issue and should no longer be a problem by the weekend

[Alfred]

I also contacted Zen Fire support recently re data problems and received their reply this morning...and have posted it below with their approval...

We have found an issue with bottlenecks getting out of the data center when
news releases cause volatility to spike. This weekend we'll be making the
changes needed to eliminate those issues and get back to normal.

The Ninja tick lag indicator is dependent on your computer clock so it isn't an
accurate gauge. I rely on the timestamps from the exchange and servers
where we control the time synchronization. We will be accurate to within a
few milliseconds and easier to track than relying on a machine running
Ninja.

We don't think any lags are acceptable but will warn traders that they are
still vulnerable to their local Internet connection and computer keeping up.
In your case, the issue is not on your end and will be addressed tonight.
(Friday night)

Another important item to note is that it didn't affect all users. Only about 1/3
of the users connected
were on the affected paths (and it only affected users connecting through
the Internet) making it less obvious when the problems began to appear. We
implemented new methods to track the presence of potential bottlenecks going
over the Internet and should be able to prevent them in the near future.