Do I still need to use a 2nd (smaller) time seres in NT8?

[NinjaTrader_Matthew]

There is no BIP1 in the context of High order fill resolution or tick replay. All OnBarUpdate events are driven from the primary series you have selected from the strategy, so if (BarsInProgress == 1) would never get hit.

If you decide to use High resolution or Tick replay, and you have intrabar logic you need to calculate, you can do so in one of the order events, such as OnOrderUpdate, or OnPositionUpdate to set/release values before the next OnBarUpdate call.

Using tick replay gives you the advantage of telling the strategy to run Calculate.OnEachTick and you could then take care of your entry logic, manual trailing stops, profit verifications, etc just like the strategy was running in real-time on each tick.

[Calonious]

There is no BIP1 in the context of High order fill resolution or tick replay. All OnBarUpdate events are driven from the primary series you have selected from the strategy, so if (BarsInProgress == 1) would never get hit.

If you decide to use High resolution or Tick replay, and you have intrabar logic you need to calculate, you can do so in one of the order events, such as OnOrderUpdate, or OnPositionUpdate to set/release values before the next OnBarUpdate call.

Using tick replay gives you the advantage of telling the strategy to run Calculate.OnEachTick and you could then take care of your entry logic, manual trailing stops, profit verifications, etc just like the strategy was running in real-time on each tick.

Just to clarify with Tick Replay -
Does this also call OnMarketData() historically as well? I've been noticing some calculations being done in when first loading the indicator on the chart

[NinjaTrader_Matthew]

Yes - OnMarketData is called when using Tick Replay.

[molecool]

Okay, that's quite a change from how it works now - let's take it from the top:

There is no BIP1 in the context of High order fill resolution or tick replay. All OnBarUpdate events are driven from the primary series you have selected from the strategy, so if (BarsInProgress == 1) would never get hit.

Okay, got it - BarsInProgress would never get called on series 1 as it doesn't exist. So in live mode the method gets called on every tick. If I set it to high resolution or tick replay it also gets called on those intervals in historical mode. So far I follow.

If you decide to use High resolution or Tick replay, and you have intrabar logic you need to calculate, you can do so in one of the order events, such as OnOrderUpdate, or OnPositionUpdate to set/release values before the next OnBarUpdate call.

I'm not so sure that's completely clear to me. Let's use an example: If I want to trail my stop manually (not using a NinjaTrader tailing stop but a manual one) on a particular threshold then I need to wait for price to descend or rise to that threshold. Based on what you're saying I would need to keep that code in my OnBarUpdate method. My OnOrderUpdate method wouldn't be called until something changes any of my order objects. I hope that's clear...

Using tick replay gives you the advantage of telling the strategy to run Calculate.OnEachTick and you could then take care of your entry logic, manual trailing stops, profit verifications, etc just like the strategy was running in real-time on each tick.

Right, that's how I understand it - so if I require manual management of price action then I simply keep that in my OnBarUpdate method - without the BIP1 clause.

Sorry for beating this to death - I want to make sure I understand this properly.

[NinjaTrader_Matthew]

No problem I understand these are new concepts and there is some internal complexities between each mode. The mode you choose ultimately depends on your needs.

Historical fill processing does not call OnBarUpdate for each tick. This *only* used for the execution of orders. If you're using High resolution, your OnBarUpdate method is still only called at the close of the primary bar. What this enables for you is the ability to process executions within the bar.

If you have logic where you need to use a trailing stop tick per tick, you are correct you would need to use OnBarUpdate() logic to have any sort of update at the frequency you require. Under the NinjaTrader 8 framework, you High resolution fill process does not help here. You would only be able to accomplish this by using Tick Replay with Calculate.OnEachTick or by the old way of manually Adding a series via AddDataSeries().

To summarize:

• If you went Tick Replay route, you would not need to do any BIP filtering-> this is just like running your strategy live (more resource intensive) and the OnBarUpdate calls depend on your "Calculate" setting.
• If you went high order fill resolution, your 480-minute series is only called once for each bar-> your executions now have tick data which allows them to execute intrabar.
• If you went the old fashioned way, your BIP logic would work the same as you're used to

[molecool]

No problem I understand these are new concepts and there is some internal complexities between each mode. The mode you choose ultimately depends on your needs.

Thanks - this is an important new feature for me and I need to understand it fully as to not embark on the wrong route when I begin refactoring for NT8.

Historical fill processing does not call OnBarUpdate for each tick. This *only* used for the execution of orders. If you're using High resolution, your OnBarUpdate method is still only called at the close of the primary bar. What this enables for you is the ability to process executions within the bar.

So you get the execution at the right time DURING the bar although the bar is only called once in historical mode. I take it that this is done at the end of the bar? If I log a timedate object - what would be the print? The first tick of the next bar?

Also, what is considered the 0 bar in this mode? In NT7 I had to write some code that called the bars differently given whether or not it was historical or live (another flag I had to wrestle with but I digress). In historical mode for instance my 1440 minute Closes[0][0] bar would print 3/15/2015 17:00:00 while my Closes[1][0] bar would print the same.

In live mode I would get 3/16/2015 17:00:00 and my Closes[1][0] bar would print 3/15/2015 17:00:01 or something. The 1-series was behind while in historical mode the 1-series was ahead or equal. This made looking at the right candle rather tricky.

Not to spin your head (hehe - it does it for me still) my question is how I will refer to candles now. If I want to read a particular candle will it behave live - meaning is the 0 candle the current/ongoing candle? Or is it the completed candle - meaning the main series is behind?

Again in other words - we are in the midst of the 15:00 candle right now. I want to process the 12:00 candle. Do I simply go back four candles or three?

If you have logic where you need to use a trailing stop tick per tick, you are correct you would need to use OnBarUpdate() logic to have any sort of update at the frequency you require. Under the NinjaTrader 8 framework, you High resolution fill process does not help here. You would only be able to accomplish this by using Tick Replay with Calculate.OnEachTick or by the old way of manually Adding a series via AddDataSeries().

I think I'm getting this now. High processing is only related to entry logic and fill processing. It does not increase the amount of times the OBU method is being called. So it does not simulate adding a second higher frequency series. Please correct me if I am mistaken ;-)

To summarize:

• If you went Tick Replay route, you would not need to do any BIP filtering-> this is just like running your strategy live (more resource intensive) and the OnBarUpdate calls depend on your "Calculate" setting.
• If you went high order fill resolution, your 480-minute series is only called once for each bar-> your executions now have tick data which allows them to execute intrabar.
• If you went the old fashioned way, your BIP logic would work the same as you're used to

Tick replay route - got it - sounds like this is what I need.
High order fill - only related to fill logic and only called once - got it.
Old fashioned still works - that's what I assumed. Good way to get started.

Does old fashioned way work a bit faster now due to multi-threading?

Thanks again for your patience in drilling this into my bony skull.

[NinjaTrader_Matthew]

I'll get back to you on your specific questions tomorrow. Thanks.

[NinjaTrader_Matthew]

So you get the execution at the right time DURING the bar although the bar is only called once in historical mode. I take it that this is done at the end of the bar? If I log a timedate object - what would be the print? The first tick of the next bar?

If you print a Time[0], it would be end of bar. If you logged an execution.Time object you captured from OnExecution, it would give the precise time of the execution based on the tick data.

For example, in OnBarUpdate() you printed

DateTime exeTime = DateTime.MinValue;
protected override void OnBarUpdate()
{		
	Print(Time[0] + " : " + exeTime);			
	
}
protected override void OnExecutionUpdate(Cbi.Execution execution, string executionId, double price, int quantity, 
	Cbi.MarketPosition marketPosition, string orderId, DateTime time)
{			
	exeTime = execution.Time;
}

Your output would be: 4/2/2015 7:30:00 AM : 4/2/2015 7:29:58 AM

Also, what is considered the 0 bar in this mode? In NT7 I had to write some code that called the bars differently given whether or not it was historical or live (another flag I had to wrestle with but I digress). In historical mode for instance my 1440 minute Closes[0][0] bar would print 3/15/2015 17:00:00 while my Closes[1][0] bar would print the same.

In live mode I would get 3/16/2015 17:00:00 and my Closes[1][0] bar would print 3/15/2015 17:00:01 or something. The 1-series was behind while in historical mode the 1-series was ahead or equal. This made looking at the right candle rather tricky.

If by 0 bar, meaning the index value (e.g., Time[0]), then it would print the end of the bar, then in the case of using High fill processing mode, it would always be the ending bar value (17:00:00)

If you're using Tick Replay with Calculate.OnEachTick, then it would correspond with the ticks, just like you saw in live... which would give you 17:00:01, 17:00:02, and so on.

If I want to read a particular candle will it behave live - meaning is the 0 candle the current/ongoing candle? Or is it the completed candle - meaning the main series is behind?

Again in other words - we are in the midst of the 15:00 candle right now. I want to process the 12:00 candle. Do I simply go back four candles or three?

Candles are still marked at the end of the bar. If the current bar is 15:00 and you want to get the 12:00, you would go back 3 candles.

Does old fashioned way work a bit faster now due to multi-threading?

If you're talking backtesting/optimizing - yes.

If you're talking about having to wait to do other things on the UI because it's thread is busy processing your strategy - yes.

If you mean the sheer amount of time your strategy calculates - no.

[molecool]

Sorry for the late response

Thanks for the elaborate response, Matthew - and sorry for the delay, I have been super busy. I think I'm wrapping my mind around the two concepts now:

Tick Replay with Calculate.OnEachTick = Historical runs like it would in live mode.
High Fill Processing Mode = Bars get called only once in historical mode but executions now have tick data.

Follow up questions:

1) Wouldn't it suffice to run Tick Replay with Calculate.OnPriceChange?

2) How does HFPM allow executions to get receive data? That's unclear to me. If OBU is called only once then how is it that executions receive the proper time stamp and price in historical mode?

3) Please tell me how the following scenarios would play out:

Strategy on 60 minute chart in TickReplay at 3:24:30pm calling OnBarUpdate():

Historical Mode:

Print(Time[0]);

Live Mode:

Print(Time[0]);

Strategy on 60 minute chart in HFPM at 3:24:30pm calling OnBarUpdate():

Historical Mode:

Print(Time[0]);

Live Mode:

Print(Time[0]);

[koganam]

Thanks for the elaborate response, Matthew - and sorry for the delay, I have been super busy. I think I'm wrapping my mind around the two concepts now:

Tick Replay with Calculate.OnEachTick = Historical runs like it would in live mode.
High Fill Processing Mode = Bars get called only once in historical mode but executions now have tick data.

Follow up questions:

1) Wouldn't it suffice to run Tick Replay with Calculate.OnPriceChange?

2) How does HFPM allow executions to get receive data? That's unclear to me. If OBU is called only once then how is it that executions receive the proper time stamp and price in historical mode?

3) Please tell me how the following scenarios would play out:

Strategy on 60 minute chart in TickReplay at 3:24:30pm calling OnBarUpdate():

Historical Mode:

Print(Time[0]);

Live Mode:

Print(Time[0]);

Strategy on 60 minute chart in HFPM at 3:24:30pm calling OnBarUpdate():

Historical Mode:

Print(Time[0]);

Live Mode:

Print(Time[0]);

Add a CurrentBar print to the Time prints, and it might be clearer, as you will see the multiple TimeStamps that correspond to each of the CurrentBar stamps.

[NinjaTrader_Matthew]

1) Wouldn't it suffice to run Tick Replay with Calculate.OnPriceChange?

Yes, you could run OnPriceChange if you don't need to updat with every single tick. This should save on CPU costs.

2) How does HFPM allow executions to get receive data? That's unclear to me. If OBU is called only once then how is it that executions receive the proper time stamp and price in historical mode?

The tick data is there for fill engine. This allows your orders to execute intrabar without it having to call OnBarUpdate.

Think of it like running your strategy live in real-time using Calculate.OnBarClose.

Even though your strategy is not running for each tick, if you have any pending order with your broker, it will fill as soon as price reaches that order, regardless of your strategy calculate setting.

3) Please tell me how the following scenarios would play out:

This depends on your calculate setting. I'd also follow the previous posters comments that printing it with current bar values should help you distinguish what is occurring here.

[molecool]

In addition, to help you with syncing your strategies after a restart, we have IsAdoptAccountPositionAware strategies concept, which will sync your "strategy" position to your real-world account position...meaning if you start your strategy and its Position is calculated flat, but your Account is long, your strategy will start "long".

Hey guys - it's been a few months but as I'm starting to convert things over now I would like to follow up on this thread with a few thoughts.

The AccountAccountPositionAware option is based on which criteria exactly? I would hazard to guess that it is based on the following:

• Symbol
• Interval
• Account

A bit more details would be appreciated or please point me to the proper documentation. Also, if my strategy determines it is flat (correctly or incorrectly) it would not set a stop for that open position, unless that stop had already been placed prior to restart/crash.

Am I correct in assuming that on AAPA being true that prior stops/targets would NOT be canceled out? That has been a problem for me with NT7 - unless I managed to relaunch the strategy with the very same entry the existing stop was canceled (with NT7 strategy syncing set to true) and when inconsistencies arose my stop wasn't there anymore.

This is a huge issue for me - being able to recover from crashes or relaunches. As my NT7 strategies are running with COBC set to false and use a 1-min 2nd series for recreating historical entries I regularly see differences, meaning entries are slightly different or are not taken at all. Especially when b/a spreads may have triggered an entry in live situations those same entries do not replicate upon reload as the 1-min execution series does not provide sufficient detail. FYI - I refrained from using ticks as my 2nd series as it increases load times and memory usage considerably.

Your continued insights would be much appreciated.

[molecool]

Little Epiphany

Something just occurred to me, so let me paint the picture to convey what I'm thinking:

• I am porting an NT7 strategy that uses a 1-min series for historical execution approximation.
• It uses manual trailing stop logic which is mostly embedded in BIP == 1 conditions.

Based on what has been discussed previously (thanks again for the insights) it's clear ot me that Tick Replay is the right option to go for NT8. The only thing that bothers me is that it's going to be a LOT more processing intensive as it churns through every single tick and spread. So back testing with a strategy in that mode will prove to be extremely time intensive beyond more than a few days of data.

Now the perfect scenario for me would be if still could retain the option to have my strategy back test on a 1-min series to cover large periods, whilst enjoying the option of restoring a strategy state in live mode exactly to pre-crash/relaunch conditions. But it just occurred to me that the use of a 2nd time series does not preclude me from running a strategy in Tick Replay mode. Therefore following the old NT7 dual series approach seems to be the right way to go. If I want perfect historical execution then I simply turn TickReplay on and I should get accurate execution based on the data available.

Again, apologies if I am beating a dead horse to death. I just want to make sure I cover all angles beforehand as my strategies are comprised of > 8000 lines of code. So I only want to refactor all that once if possible ;-)

[NinjaTrader_Matthew]

Hey guys - it's been a few months but as I'm starting to convert things over now I would like to follow up on this thread with a few thoughts.

The AccountAccountPositionAware option is based on which criteria exactly? I would hazard to guess that it is based on the following:

• Symbol
• Interval
• Account

A bit more details would be appreciated or please point me to the proper documentation. Also, if my strategy determines it is flat (correctly or incorrectly) it would not set a stop for that open position, unless that stop had already been placed prior to restart/crash.

Am I correct in assuming that on AAPA being true that prior stops/targets would NOT be canceled out? That has been a problem for me with NT7 - unless I managed to relaunch the strategy with the very same entry the existing stop was canceled (with NT7 strategy syncing set to true) and when inconsistencies arose my stop wasn't there anymore.

This is a huge issue for me - being able to recover from crashes or relaunches. As my NT7 strategies are running with COBC set to false and use a 1-min 2nd series for recreating historical entries I regularly see differences, meaning entries are slightly different or are not taken at all. Especially when b/a spreads may have triggered an entry in live situations those same entries do not replicate upon reload as the 1-min execution series does not provide sufficient detail. FYI - I refrained from using ticks as my 2nd series as it increases load times and memory usage considerably.

Your continued insights would be much appreciated.

IsAdoptAccountPositionAware = true under the hood is allows the user to select "StartBehavior.AdoptAccountPosition" and then the start behavior will act based on the rules defined here:

http://ninjatrader.com/support/helpGuides/nt8/en-us/index.html?syncing_account_positions.htm#AdoptAccountPosition

When that is set, the Strategy will automatically disregard whatever position the strategy was "calculated" to and will just use the current position of the Account. However this doe assume that the developer of the strategy is aware of the PostionsAccount interface which allows you to place/check orders based on the real-world account rather than the calculated position, and you may find yourself in some scenarios that you should have a stop/target but did not if you have not checked against the PositionsAccount. But that should not be a problem if your strategy had first placed some protective orders and then was later restarted-> those orders should be mapped back to the strategy.

If your strategy is having issues mapping to the real-world orders with this setting, I would have to wonder why is happening between the time the strategy has stopped and is re-started that it cannot match those orders. If you're manually modifying/canceling/resubmitting orders, it could certainly break this logic. If that is part of the game plan, then you should consider making your strategy more "Account" aware by using the Account class and checking real-world accounts for existing orders and positions and not relying on the historical processing if that is what is causing problems for you

[molecool]

IsAdoptAccountPositionAware = true under the hood is allows the user to select "StartBehavior.AdoptAccountPosition" and then the start behavior will act based on the rules defined here:

http://ninjatrader.com/support/helpGuides/nt8/en-us/index.html?syncing_account_positions.htm#AdoptAccountPosition

When that is set, the Strategy will automatically disregard whatever position the strategy was "calculated" to and will just use the current position of the Account. However this doe assume that the developer of the strategy is aware of the PostionAccount interface which allows you to place/check orders based on the real-world account rather than the calculated position, and you may find yourself in some scenarios that you should have a stop/target but did not if you have not checked against the PositionAccount. But that should not be a problem if your strategy had first placed some protective orders and then was later restarted-> those orders should be mapped back to the strategy.

If your strategy is having issues mapping to the real-world orders with this setting, I would have to wonder why is happening between the time the strategy has stopped and is re-started that it cannot match those orders. If you're manually modifying/canceling/resubmitting orders, it could certainly break this logic. If that is part of the game plan, then you should consider making your strategy more "Account" aware by using the Account class and checking real-world accounts for existing orders and positions and not relying on the historical processing if that is what is causing problems for you

First up thank you for addressing my question. In case you missed it - I posted a 2nd one after that (#28). Would you mind covering that one as well?

I read the section covering AdoptAccountPosition section and understand that preference is given to your account position. So that setting would set my strategy to LONG if my account was already long, even if my strategy thinks after reload that I actually should be SHORT. The section does not mention that I internally also need to verify my account via the PostionsAccount interface in order to be able to set the proper stop and target. Assuming I understand this correctly that is an important implication and there should be mention of PostionsAccount in the documentation.