Understanding states and data persistence

[DerkWehler]

I am trying to understand how to setup a ninjascript strategy. I have found many things that ought to be in the ninjascript documentation, but is not...

There is the State.SetDefaults for inputs and State.Configure for globals for basic use, I understand.

But I am wondering about data persistence; particularly if I have to restart NT8 for any reason, including a where State.Terminated was not called (e.g. computer crash).

Easiest to use an example:
Suppose I have a strategy that has an input "MinDailyProfit". If set to zero, it is disabled, but suppose it is set to $500.

In OnExecutionUpdate() I can check how much the Strategy has made today, using:

SystemPerformance.AllTrades.TradesPerformance.NetP rofit

So...first question is, what net profit does that return? Is it only trades place by THIS strategy, and for today (and is it by session actually, not day)?

In State.SetDefaults, I have a variable, pDaily, set that way:

pDaily = SystemPerformance.AllTrades.TradesPerformance.NetP rofit;

Then in OnExecutionUpdate, I can do this:

if (Position.Quantity == 0)
{
    if (DailyMinTarget > 0 && SystemPerformance.AllTrades.TradesPerformance.NetProfit - pDaily > DailyMinTarget)
    {
        // These are "just in case", like if there's a reversal -to make sure I'm flat.  But may not be needed
        ExitLong("Daily Target", "Long");
        ExitShort("Daily Target", "Short");

        Print($"\n\nNetProfit (${SystemPerformance.AllTrades.TradesPerformance.NetProfit}) - pDaily (${pDaily}) > DailyMinTarget (${DailyMinTarget}");
        Print($"Daily Profit Hit! Set dailyLimitHit = true\n");

        dailyLimitHit = true;
    }
}​

Assume the profit target was hit and dailyLimitHit=true (which I use to disable further trade).
It is my understanding that if I have to restart, that even w/o my pDaily starting amount, NT runs through historical data and would end up setting dailyLimitHit back to true based on that(?)

If so... what about if my strategy uses a proprietary signal which cannot work on history?
That being the case, in OnBarUpdate() I may as well have this at the top:

if (State != State.Historical)
return?

Correct? And in that case, my use of pDaily should handle this case and re-set dailyLimitHit = true(?)

Is there some tutorial somewhere that teaches exactly how NT calls the strategy with each different state change, like what order they are called?

And perhaps how to deal with data persistence (saving and reading back in variable values)?

[NinjaTrader_Jesse]

Hello DerkWehler,

The platform will only save values for public user inputs, if you change a user input using the user interface and save a template or the workspace that will save its current value for the applied instance. Any other value that you wanted to save you would have to do that yourself by writing that data to file. The user inputs are used as a starting place for values, in your example the starting place is 500$. If your logic can historically calculate the same orders again you should reach the same result that you previously had using that as the starting point. If your logic cannot calculate historically then that would be a realtime only strategy which in general would not be able to use any kind of historical processing to re calculate the value so it would start at 500$.

You can read about the lifecycle of scripts in the following link:

https://ninjatrader.com/support/helpGuides/nt8/understanding_the_lifecycle_of.htm

Regarding what net profit returns, that returns the net profit the same named value that you see in the strategies performance report.

[DerkWehler]

In setting global member variables in the

if (State == State.Configure)

section of OnStateChange(), I have this:

pDaily = SystemPerformance.AllTrades.TradesPerformance.NetProfit;
Print($"pDaily initialized to: ${pDaily}. State = {State.ToString()}");

..and this printed in the log:

pDaily initialized to: $0. State = Configure
pDaily initialized to: $0. State = Configure
pDaily initialized to: $0. State = Configure
pDaily initialized to: $0. State = Configure

Why would this be called multiple times with the same state?!

[NinjaTrader_Jesse]

Hello DerkWehler,

State.Configure would not be a valid location for that type of code because it relies on OnBarUpdate to process and have orders submitted. The AllTrades collection contains each paired entry+exit and derives the performance from that information. If your strategy is realtime only that won't have a value until after the first entry+exit has been filled in realtime. If your strategy can process historically it will populate after the first historical entry+exit but would still need to be used from OnBarUpdate.

The State.Configure is called for multiple reasons and would depend on the specific actions that you completed in the user interface, multiple instances of the script will be created before its enabled. To do something once before bar processing you can use State.DataLoaded assuming that code does not need to go in OnBarUpdate, that would mainly be useful for setting initial variable values. Anything that relies on orders being filled or bar processing needs to go in OnBarUpdate.

[DerkWehler]

Thank you for the reply, Jesse.

The platform will only save values for public user inputs, if you change a user input using the user interface and save a template or the workspace that will save its current value for the applied instance. Any other value that you wanted to save you would have to do that yourself by writing that data to file. The user inputs are used as a starting place for values, in your example the starting place is 500$. If your logic can historically calculate the same orders again you should reach the same result that you previously had using that as the starting point. If your logic cannot calculate historically then that would be a realtime only strategy which in general would not be able to use any kind of historical processing to re calculate the value so it would start at 500$.

Well, actually, the MinDailyProfit would always stay at $500, but I get the idea; the strategy (thru history) would be unable to get signals to reach the conclusion that the $500 limit had been hit.

But in that case, the pDaily variable should handle it(?) Though I still don't see why it's set 4 times (see last post).

You can read about the lifecycle of scripts in the following link:

https://ninjatrader.com/support/helpGuides/nt8/understanding_the_lifecycle_of.htm

Regarding what net profit returns, that returns the net profit the same named value that you see in the strategies performance report.

Okay, thanks, I will read up on that. The NT performance report is fraught with errors (delays, duplicated trades), but I get the idea.

However, are you saying it returns the net profit for the current day/session (as a sort of default), or that it depends on what time period I have set, because of course that wouldn't make any sense if I did not have a performance window open...

[NinjaTrader_Jesse]

Hello DerkWehler,

If MinDailyProfit is a user input then yes it will stay at a constant value. Your other variables would rely on bar processing to populate, the pDaily variable would need to be in OnBarUpdate to see that new trades have been added to the collection so the value you set to it has a value. If you don't allow historical processing it will start at 0 until a realtime entry+exit has been filled.

The AllTrades collection includes all trades for all sessions, if the strategy processed 10 sessions historically it will have the values for all 10 sessions. if you do not allow historical processing it will start at 0, if the strategy stays enabled into the next session it will also include the previous sessions trades. There is a sample of how to work with that collection on a per session basis here: https://ninjatrader.com/support/help...nce_statis.htm

[DerkWehler]

Hello DerkWehler,

State.Configure would not be a valid location for that type of code because it relies on OnBarUpdate to process and have orders submitted. The AllTrades collection contains each paired entry+exit and derives the performance from that information. If your strategy is realtime only that won't have a value until after the first entry+exit has been filled in realtime. If your strategy can process historically it will populate after the first historical entry+exit but would still need to be used from OnBarUpdate.

The State.Configure is called for multiple reasons and would depend on the specific actions that you completed in the user interface, multiple instances of the script will be created before its enabled. To do something once before bar processing you can use State.DataLoaded assuming that code does not need to go in OnBarUpdate, that would mainly be useful for setting initial variable values. Anything that relies on orders being filled or bar processing needs to go in OnBarUpdate.

Great info, thanks again. Much to read/learn.

For now I have no ability to do historical signals, but hopefully I will be eventually able to add it.

Your other variables would rely on bar processing to populate, the pDaily variable would need to be in OnBarUpdate to see that new trades have been added to the collection so the value you set to it has a value. If you don't allow historical processing it will start at 0 until a realtime entry+exit has been filled.

Okay, I think I understand. Without historical processing, it starts at zero, and would not change until a realtime entry+exit has been filled since restarting?

Whereas if there was historical, it would be the PnL so far, for the current session?

[NinjaTrader_Jesse]

Hello DerkWehler,

Yes that is correct however with or without historical processing that is just like the example I provided. Its not just the current session its however many sessions the strategy processed over and placed trades on. If historical is enabled and the strategy processes 10 sessions it will contain trades for all 10 sessions. If the strategy was not allowed historical it starts at 0 into the current session, if the strategy is left on into other sessions it will include all sessions where it placed trades.

[DerkWehler]

Hello DerkWehler,

Yes that is correct however with or without historical processing that is just like the example I provided. Its not just the current session its however many sessions the strategy processed over and placed trades on. If historical is enabled and the strategy processes 10 sessions it will contain trades for all 10 sessions. If the strategy was not allowed historical it starts at 0 into the current session, if the strategy is left on into other sessions it will include all sessions where it placed trades.

So if I want it to reset every day (lets say session, since futures run 18:00 ET to 17:00).
If it had historical, I could use this:

if (Bars.IsFirstBarOfSession)
{
    pDaily = SystemPerformance.AllTrades.TradesPerformance.NetProfit;
​}

because each new day, it would set it to the total so far, which can then be compared with total going forward.

without historical, if it runs today, hits the target, then I reset the strategy, I would need to have saved that info (how much it made, or that target was hit) somewhere and re-initialize when it starts up again. ChatGPT has given me some samples about saving state, but though good with C#, it's notoriously bad at ninjascript. Is there some article / link where I can read how to do it 'properly'?

Note: I do see you included a reference to "Using trade performance statistics for money management", where there is a sample "SamplePnL_NT8.zip" I can download. Were you thinking that sample can give me some good tips about this?

[NinjaTrader_Jesse]

Hello DerkWehler,

If you want to reset each day you can see the sample that I linked in post 6. That shows how to use per session values. The sample is listed on that page, you can click it to download the zip file. The link you provided links to that file.

The code you have shown would not reset the value, that is still the total for all sessions, the name of the collection is the hint here "All Trades" meaning every trade the strategy has placed while enabled. That value does not consider sessions at all.

Regarding ChatGPT, we highly recommend to avoid using that tool as its current state often provides invalid examples that won't actually work. Unless you are very experienced in C# and in NinjaScript that tool is pretty useless currently. If you are skilled in both C# and NinjaScript you can use it for suggestions because you at that point know which parts are not valid when it recommends something random. For learning NinjaScript that will often take you down a wrong path where you will need to start over. The reference samples in the help guide are generally the best place to start for specific concept, you can also post on the forum any questions that you have and our support can link to existing samples that relate to that topic from the help guide, forum or user uploaded scripts.

[DerkWehler]

If you want to reset each day you can see the sample that I linked in post 6. That shows how to use per session values. The sample is listed on that page, you can click it to download the zip file. The link you provided links to that file.

The code you have shown would not reset the value, that is still the total for all sessions, the name of the collection is the hint here "All Trades" meaning every trade the strategy has placed while enabled. That value does not consider sessions at all.

Got it. I'll read over the sample before going any further.

Regarding ChatGPT, we highly recommend to avoid using that tool as its current state often provides invalid examples that won't actually work. Unless you are very experienced in C# and in NinjaScript that tool is pretty useless currently. If you are skilled in both C# and NinjaScript you can use it for suggestions because you at that point know which parts are not valid when it recommends something random. For learning NinjaScript that will often take you down a wrong path where you will need to start over. The reference samples in the help guide are generally the best place to start for specific concept, you can also post on the forum any questions that you have and our support can link to existing samples that relate to that topic from the help guide, forum or user uploaded scripts.

Yes, I have learned that it is okay for getting some ideas. For example, when I asked how to save and reload values, it gave some code that, while not correct, may have pointed me in the right direction. It sees to speak with such authority, it took my a while to realize that didn't mean anything.

Really appreciate the quick responses.

[NinjaTrader_Jesse]

Hello DerkWehler,

Yes that is part of the problem with that tool, it states the infomration like it is a fact so its hard to dicern if that is actually true or not. For general C# questions it can be helpful for learning but for NinjaScript it does not have a good reference point for its learning. That pulls information from many locations like the NT7 help gude or forum which is not able to be used with NT8, it has provided invalid code based on users non working code from the internet, it can be completely random. It also makes up method or property names assuming you will replace it with valid code when you see that it is not valid.

For loading and saving data there are two samples in the help guide for that, you can locate those in the following page.

Using StreamReader to read from a text file
Using StreamWriter to write to a text file

https://ninjatrader.com/support/helpGuides/nt8/indicator2.htm

[DerkWehler]

The "lifecycle" page was some good info, but did not go into enough detail about historical bar processing. I realize now it may be all that much more complicated by multi-threading and my tendency to think about it linearly, but interested in some detail once it gets to State.Historical, regarding OnBarUpdate().

Does it call OnStateChange() once with State.Historical, then start looping through all bars (starting with oldest), calling OnBarUpdate(), at the end, "realizing" that it's "now", and then setting State to something besides Historical?

[NinjaTrader_Jesse]

Hello DerkWehler,

The lifecycle page is mainly intended to go over the states in OnStateChange, not how the script processes data. The OnBarUpdate documentation explains how OnBarUpdate is called which is where historical bars are processed. The subsequent pages in that section also cover the various properties that are used to know which bars are processing or change how bars process. The multi series documentation also has some visuals of how bars are processed and goes into more specific detail about processing and using multi series.

https://ninjatrader.com/support/helpGuides/nt8/onbarupdate.htm

https://ninjatrader.com/support/help...ithMultiTimeFr ameObjects

Regarding threading, you won't have to deal with threading in a script in most situations, the only time you would actively need to deal with threading would be with more complex code that modifies or interacts with the user interface. Your normal NinjaScript code will never use any C# threading concepts, that is all controlled internally by NinjaTrader. The methods like OnBarUpdate and OnExecutionUpdate are event driven by the data and order events so when you are in realtime those can be called independently and not in a specific order. In historical each override has a specific way it processes, you can read about each override in their respective help guide pages.

Does it call OnStateChange() once with State.Historical, then start looping through all bars (starting with oldest), calling OnBarUpdate(), at the end, "realizing" that it's "now", and then setting State to something besides Historical?

Correct, the state changes depending on where the script is in its processing.

[DerkWehler]

Great on the previous post. I will [re]read OnBarUpdate. And I have already had to deal with some threading stuff calling async methods, but good to know it's not something I need to normally be concerned with.


The SamplePnL code is great for telling me how to save info about trades up to that point. Since my strategy is unlikely to ever be running on the first bar of the session (except for historical), I need to relocate it to first bar of my session time period; no big deal.

However, assuming I still have no historical ability, if my computer crashes suddenly, then when I restart, it has no idea how to calculate how much it already made today. Strange to me how this is designed; I'm an MT4 veteran and in that, you can just iterate over all closed trades and filter by date.

So it would seem I still need the ability to save that state and reload it on start up. That code says to do this:

if (Bars.IsFirstBarOfSession)
priorTradesCumProfit = SystemPerformance.AllTrades.TradesPerformance.Curr ency.CumProfit;

...which I will do when [my trading] session starts, but I also need to save that figure somewhere and be able to read it back in at startup.

So for now, I would keep this:

protected override void OnBarUpdate()
{
    // No point in trying to do historical trades without the signals
    if (State == State.Historical)
        return;
    ...
}

..which means my:

if (BarsInProgress == 0)
{
    if (Bars.IsFirstBarOfSession)
    {
        pDaily = SystemPerformance.AllTrades.TradesPerformance.NetProfit;
    }
    ...
}
​

...will never be called unless it's running at 6pm ET. But I can make the same assignment at the start of my trading session, and then I need to save it somewhere, and read it back in if it exists.

So can you offer a link to how to do that correctly?