Ability to access former live strategy performances distinctly

[tony_28217]

I just posted elsewhere about this and understand the current capabilities here.

I understand strategy analyzers have defined starts and finishes, which may make the scenario I described difficult but it would be really useful to be have historical live strategy performance be recallable in individual analysis just the same as preset strategy analyzer runs. Simple stuck in a file so I can reopen them.

I outlined a few scenarios in my previous post but ultimately Restarting a strategy (for whatever reason) doesn't use the historical fills for that so we can't really know how well they did without much more scrutiny and effort for analysis. This makes it difficult to know. And if I'm trading multiple of them, the Trade Performance features are not quite meeting the need.



An entirely separate idea:
An optional additional version of this that could be useful is to be able integrate previous live run data (the option at least to select those runs) to test for convergence. But this may ultimately be too difficult. This would allow for "backtesting meets livetesting." Sometimes I'll stop/start certain strategies for various reasons and it would be great to aggregate them all together. Not sure if tehre is some way to make this work.

Cheers!

[NinjaTrader_ChelseaB]

Hello tony_28217,

You can write the execution object from OnExecutionUpdate() and the order object from OnOrderUpdate(), or the SystemPerformance information (such as the last trade) to a text file if you wanted to review at a later time.

Below is a link to an example that writes information to file.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance#post100192

https://ninjatrader.com/support/helpGuides/nt8/using_streamwriter_to_write_to.htm

However, you appear to be describing the Strategy Analyzer logs which saves each backtest or optimization so this can be re-opened with the same parameters.

https://ninjatrader.com/support/helpGuides/nt8/backtest_logs.htm

Where you have mentioned:
" ultimately Restarting a strategy (for whatever reason) doesn't use the historical fills"

When restarting a strategy, the strategy will process the historical data and if the same orders are submitted with the same price and time the orders are resumed (and marked as realtime). 1-tick intra-bar granularity helps for these to be resumed. Below is a link to a forum post that discusses.

https://ninjatrader.com/support/forum/forum/historical-beta-archive/version-8-beta/87750-load-cumulative-profit#post783182

Where you have mentioned:
"integrate previous live run data "

NinjaTrader intentionally only has orders made by the instance of the strategy, as this is the only accurate way to for the strategy to process data and orders.

You could choose to read information from a text file, and use that to submit orders at specific times, and / or use AdoptAccountPosition to set the position to something not calculated by the strategy.

[tony_28217]

Hi Chelsea,

Thanks for the thorough response.

I know many things can be taken out of NT as text or otherwise.

I guess the disconnect is this:

I can view (as they are saved by the program) any older view of an NT Strategy in the Backtest/OPtimization stage. If appreciably the same UI is used for Strategy Performance as for Strategy Analysis - when you right click the strategy > Strategy Performance > Real Time - why can't I view any previous runs of that strategy within it once they become 'previous runs' of the strategy? Surely there would be a way to log this file similarly and allow it to recalled/reloaded into the analysis tab?


Sorry for adding too much in the previous previous post about aggregating old runs. I really only care to be able to recall and reload an individual FORMERLY live run into the NT desktop for easier comparison.

Thanks!

[NinjaTrader_ChelseaB]

Hello tony_28217,

The historical data is not duplicated and saved (as this would fill up hard drives very quickly and would increase loading speeds considerably) but the parameters and results are saved to file.

Take for instance, if you were to add that to a chart that has different historical data loaded and indicators were added to the chart with AddChartIndicator() and the values are used for logic in the script. Would the indicator somehow be applied to the chart bars but not using those chart bars for the calculations? Meaning the indicator wouldn't match the data on the chart it's displayed over.

To confirm, are you requesting the real-time or historical data itself be duplicated and saved with the backtest results so that this is used for the calculations when its started live?
Or are you asking for price and indicator values to stored, and not recalculated with the charts loaded data? (which can cause discrepancies)

[tony_28217]

Hi Chelsea,

Yeah I see the confusion. I was trying to put 2 requests in one post. That is my mistake.

I think disregarding the concept of loading historical data to be used for future calculation would be best to complete the intended request.

The point I'm trying to get at for the most impact is the storing (for later recall) of real-time strategy information that can be reloaded SOLELY for analysis back into NT not for recalculation on restart. As I tried to say, I irreconcilable differences without many more parameters so please ignore the bolded part.


I just want to know how well a particularly strategy has performed even if it is started, stopped. Best Practices in no 12 suggests restarting NT Desktop daily. But if I did that, the strategies real time performance would be difficult to compare. This is irreconcilable, no? And not to mention when an inadvertent restart might occur.

If there is a way to log this information and RELOAD it into NT desktop, that is ideal. That way we can actually compare all RealTime result of a strategy to Historical ones.

Thanks!

[NinjaTrader_ChelseaB]

Hello tony_28217,

If a strategy is run in real-time, disabled, then re-enabled, and is setup to resume, this is going to have all of the real-time trades, even from the previous run and will have a total calculation (as if it was never disabled at all).

If you want to compare the real-time to historical, just add a new instance or test in the strategy analyzer.
Comparing real-time, historical, and replay performance - NinjaTrader Support Forum

To confirm, you specifically want this saved to a file so that you can generate new instances with the same indicator values and order collection without saving the data and allowing these to not match the data on the chart?

[tony_28217]

Hey Chelsea,

Sorry. I think I'm confused. I just stopped and started a strategy that had been running with multiple trades for the past few days. When I go into Strategy Performance > Select that Strategy > Real Time, it shows only one trade that occurred. I had it set to Adopt Position if that matters.

In the way you described it, shouldn't I see all those previous trades in the Real Time performance?

You did comment: "setup to resume" is that a particular setting or something else I need to do?


And no To confirm, I DO NOT want to generate new instances from this data. Solely retroactive analysis of past (previous run and then stopped strategy's) real time performance.

[NinjaTrader_ChelseaB]

Hello tony_28217,

May I confirm you have reviewed the forum post linked in post # 2 on resuming a strategy. Resuming can only be done with the Immediately Submit start behavior with 1-tick intra-bar granularity implemented (and TickReplay if running with Calculate.OnEachTick/OnPriceChange).

If you have Adopt Account position, this will not resume the previous orders.

[tony_28217]

Hi Chelsea,

I wanted to let the strategy run overnight to ensure some data would be run.

1. I am using Calculate.OnPrice Change
2. I have changed over to Immediately Submit and Synchronize per your comment.
3. I use IsTickReplay = True (the chart is loaded with tick replay on) and intrabar granularity is on as well with AddDataSeries of 1 Tick set.

I also saved the workspaces as you outlined and further reviewed the other posts you have made to others and followed them step by step.

Before closing: The Real Time Performance had trade information. The summary is populated as you'd expect while running

Upon Restart of the strategy: The Real Time Performance is empty.

Should I be looking at it in some other way?

Thanks again for all the help!

[NinjaTrader_ChelseaB]

Hello tony_28217,

Intra-bar granularity means submitting the orders with the barsInProgress index of the added 1 tick series in the order method. May I confirm you are doing this?

If the orders are not matching, they cannot be resumed.

Further, Immediately Submit must be set for a new instance.
"To do this, you will need to enable the following option before starting a new instance of a strategy:"

https://forum.ninjatrader.com/forum/historical-beta-archive/version-8-beta/87750-load-cumulative-profit?p=783182#post783182

Please write to file the order object from OnOrderUpdate() so that we may confirm the order fills are the same during the real-time orders, and after the script is reloaded.
Include the output with your next post.

[tony_28217]

Hi Chelsea,

Intra-bar granularity means submitting the orders with the barsInProgress index of the added 1 tick series in the order method. May I confirm you are doing this?

Yes. I have done this.

If the orders are not matching, they cannot be resumed.

And this is where we have a problem. If there is not convergence from backtest to real time (which my strategy sometimes struggles with) we basically arrive back at the beginning which is the inability to review prior performances.

Is it possible to submit a development request that would simply allow a strategy's performance, on it's ending, to be able to be saved in a file that can be reloaded into the strategy analyzer? That's really all I want here.

Thanks again

[NinjaTrader_ChelseaB]

Hello tony_28217,

This is not what the Strategy Analyzer was designed for, so this request may not gain much traction. However, if you would like to submit this request without further investigating to find an immediate solution, please let me know and I will submit the request.

The Strategy Analyzer is specifically used for running a strategy through a backtest or optimization using the data that is available.

We should look further into what is causing differences. May I have the output from printing the tick, condition, and order information so that I may review? (As demonstrated in the RealtimeReplayHistoricalComparisonsExample_NT8)

[tony_28217]

I guess I really don't mind what it's called but really the term Strategy Performance (which seems to be essentially the same layout as the analyzer). But I can see that there isn't much movement here.


As for the "cause of the differences" I do see the issue.

On Backtests - despite downloading historical tick data, having 1 tick of accuracy and running Tick Replay and addressing OnMarketData (all of which I believe are the requirements from my research), and say I run with a primary data series of 30 minutes, all my fills happen at the 00/30 of the hour, while clearly that is not the case in real time.

[NinjaTrader_ChelseaB]

Hello tony_28217,

OnMarketData does not run in historical (backtest) unless TickReplay is enabled.

If orders are being filled on bar close, then the script is not submitting the orders to the BarsInProgressIndex of the 1 tick series, or in the case of market orders, are not being submitted intra-bar with TickReplay and Calculate set to OnPriceChange/OnBarClose.

What BarsInProgress index is being supplied to the order methods?

[tony_28217]

Ah yes. That'll do it. It got all rearranged when trying to solve for a few other problems. THANKS!