Please improve performance using Range bars

[NinjaTrader_Dierk]

Great, thanks for clarification.

[JS999]

Are you running multiple NT instances in different VMs against the same <my documents>/NinjaTrader 6.5 directory?

No... each virtual machine is a completely separate instance of Windows XP, with it's own hard disk structure. A separate copy of NT is installed inside each machine.

[NinjaTrader_Dierk]

Are you running multiple NT instances in different VMs against the same <my documents>/NinjaTrader 6.5 directory?

>> Having said that, my question for NT support is: if I have a bunch of charts open for an instrument, and if the NT historical data server goes down, will I not be able to see that data any more? I figure as long as I don't close the chart, there shouldn't be a problem with viewing whatever I've got open, since the data should just be in memory (obviously if I close the chart down I won't be able to see it). Is this correct?
Your understanding is correct

[JS999]

Alright... so after talking with Mirus tech support today, I've got some more information on this. Apparently turning the real time bar data option off should not affect things in terms of data integrity - I guess the first guy who told me that was just wrong. However, they did say that if the NinjaTrader historical server goes down and I don't have this option checked, I won't be able to see any historical data for my charts (obviously).

Having said that, my question for NT support is: if I have a bunch of charts open for an instrument, and if the NT historical data server goes down, will I not be able to see that data any more? I figure as long as I don't close the chart, there shouldn't be a problem with viewing whatever I've got open, since the data should just be in memory (obviously if I close the chart down I won't be able to see it). Is this correct?

Secondly, just as a reference, Mirus tech support told me to turn off my firewall and antivirus software completely in order to get the best performance... has anyone else noticed that antivirus/firewalls can cause obvious slowdowns?

[NinjaTrader_Dierk]

JS999
>>All that it means is that I am running multiple copies of NT
I'm not familiar with VM setups, but if all NT instances run against the same <my documents>/NinjaTrader 6.5 directory, then this definitely is not supported. That type of setup could cause concurrency issues as multiple instances access the same data set. That's why you can't start multiple instances of NT6.5 at a time. I suggest not trying to trick out the limitations we purposely coded.

This could be the reason why you are getting yourself into trouble.

[NinjaTrader_Ray]

I can't say that it would be responsible however, for sure the more instruments you subscribe to and an influx of data will incur some expense. Since the "Store real time bar data" is not needed, I would disable it and see if it makes a difference.

Store real time bar data is simply a recording/flushing mechanism to provide users who don't have access to historical data a means to capture and store it.

[JS999]

If using Zen-Fire or any other broker that offers historical data, there is never a need to have this enabled. It will only eat up CPU cycles. This is run on a separate thread, buffered and flushed to disk once a minute. I recommend that you uncheck this, its not needed.

Alright... I haven't been able to get in touch with Mirus tech support to confirm this yet, but do you think that this could be the problem when I see a sudden burst of data that comes in over the connection, in terms of the chart not updating? In other words, is it possible that the system is busy doing something with that storing of bar data, and that's why the chart can't update before it's finished?

The same condition results in simulation trades being "held up" until the data burst is finished... in other words, nothing happens, no user input is processed, and the system is completely "locked" until the burst is finished. As someone familiar with the internals of the program, does it sound like the "store realtime bar data" option could be responsible for this?

Thanks!

P.S. I've asked this question a few times in terms of trying to track down why a data burst could tie up the system like that, but never really got a definitive answer so far...

[NinjaTrader_Ray]

You have "store realtime bar data" turned off? Mirus tech support told me in no uncertain terms that this needed to be on in order to function and display prices properly... maybe that's why your performance isn't lagging.

If using Zen-Fire or any other broker that offers historical data, there is never a need to have this enabled. It will only eat up CPU cycles. This is run on a separate thread, buffered and flushed to disk once a minute. I recommend that you uncheck this, its not needed.

[JS999]

I have that unchecked per Ninja's recommendation:

Store real-time bar data

Enables or disables the storage of incoming real-time Chart or Market Analyzer data to your local PC for future historical data requests. If you are connected to a provider that supports historical data, disable this feature.

Zen-fire's limited historical data is suitable for me.

Alright.... but the Mirus rep said something about prices not being displayed properly if that wasn't checked, I believe. I don't think it was just a historical data thing. You might want to call them and double check, just to be sure... I could be the one who's wrong, but the support rep seemed pretty adamant about it.

[mrpowerballad]

I have that unchecked per Ninja's recommendation:

Store real-time bar data

Enables or disables the storage of incoming real-time Chart or Market Analyzer data to your local PC for future historical data requests. If you are connected to a provider that supports historical data, disable this feature.

Zen-fire's limited historical data is suitable for me.

[JS999]

My range bars are set to 3 day look back -- real-time data and market replay are turned off. I trade ES, NQ, YM, and TF -- no more than two instruments up at a time, no more than three charts at a time. My OS is XP, and I'm not running any sort of virtual environment -- maybe that's part of the difference (?)

Sounds like you've thoroughly tested this -- hopefully NT7 will solve the problem. Good luck.

You have "store realtime bar data" turned off? Mirus tech support told me in no uncertain terms that this needed to be on in order to function and display prices properly... maybe that's why your performance isn't lagging. I would call them and discuss it, if you do in fact have it off...

I have tried it on XP as well as Vista and Windows 7... seems to be about the same everywhere....

[mrpowerballad]

How far back are your range bars going (in terms of days)? And what instruments? That will help me to do some more testing...

My range bars are set to 3 day look back -- real-time data and market replay are turned off. I trade ES, NQ, YM, and TF -- no more than two instruments up at a time, no more than three charts at a time. My OS is XP, and I'm not running any sort of virtual environment -- maybe that's part of the difference (?)

Sounds like you've thoroughly tested this -- hopefully NT7 will solve the problem. Good luck.

[JS999]

Can you elaborate upon how you're running NT in VMs, and how that increases performance? I know this is off topic, but I've seen you mention this in other threads, and I'm just curious since I have an i7 system sitting next to me waiting to be built, and I would like to take full advantage of what it can do if possible.

All that it means is that I am running multiple copies of NT, each with its own connection to my broker. This means that I can assign the entire VM to one of the cores, so in effect, if I am running 3 of these things then I get to use 3 of the 4 cores of my i7 920 system. Each copy runs a number of charts, so the load is split up amongst the cores. I make sure that the copy that runs on the core that handles actual trades only has a couple of small charts open, with the other copies running longer terms charts or charts with more indicators. One VM runs a long-term range chart which would kill my the T&S coming in if I ran it on the main copy, for example.

[Radical]

Can you elaborate upon how you're running NT in VMs, and how that increases performance? I know this is off topic, but I've seen you mention this in other threads, and I'm just curious since I have an i7 system sitting next to me waiting to be built, and I would like to take full advantage of what it can do if possible.

[JS999]

Just want to chime in to say that I also use range bars usually with 2 or 3 instruments/charts, multiple indicators on each chart set to Close=False, update interval set to 0, and I haven't had any notcible problems with lag in the year or so I've been using Ninja. My computer is very standard -- 4 gb ram, 2 core E8400 chip. I also use Zen-Fire/Mirus (via Comcast interent).

How far back are your range bars going (in terms of days)? And what instruments? That will help me to do some more testing...

Just out of curiosity, are you measuring bar lag soley by time and sales? Not to point out the obvious, but are you certain it's the case that the bursts of activity on the T&S you're associating with lag are actually pushing through new levels? If not, then the range bars may not have a reason to update. You often see huge bursts of acitivity on the ES tape, only to then notice that a particular price level was never actually broken.

Yes, they are pushing through new levels, and in a dramatic way. And it's not just the range bar charts that don't update, it's all the charts. Time + sales scrolls through at an extremely rapid rate, the chart is frozen, then all of a sudden the chart jumps 10 ticks higher.

However, if a minute bar or the DOM is pushing through new levels and the range isn't doing anything, then that's obviously a problem. Have you compared range bars to minute bars or DOM side-by-side?

I have multiple connections to Mirus, some running in virtual machines. What I have done is set up a connection through a VM and another normal one. I note that the normal one is typically faster in terms of getting the data (by a minuscule amount). However, when I add a range chart or two, all of a sudden the normal T & S slows down be slower than the one in the VM.

This has been happening across multiple OSes that I have tried it on, and more than one internet connection.