Strategy Analyzer Walk Forward Test's Aggregate Sortino Seems Incorrect

[ozziyhtomit]

Hi all,

I am confused on the Sortino shown in the first row (aggregate row) for the walk forward results. Few points:

1. For one test I ran the aggregate Sortino is 4.7.

2. When I calculate it myself on the dailies I get 3.6 and when I calculate it from the monthlies I get 3.2. These are close enough to 4.7 bc my assumptions on risk free rate and days in a trade year may be different etc.

3. Now I go to the backtester feature instead of the walk forward test. I manipulate the walk forward to match the backtester param selection so the results should be similar if not identical. In there it shows a Sortino of 1.3.

Can someone shed any light on the walk forward aggregate Sortino? Is it possible maybe each Sortino was written by a different team or something so the underlying assumptions are different? Risk free rate, number of trade days in a year etc? Other?

Thank you all,
Tim

[NinjaTrader_Jesse]

Hello ozziyhtomit,

Welcome to the NinjaTrader support forums.

You can try taking a look at the description in the help guide here to see if this matches what you are trying to manually calculate: https://ninjatrader.com/support/help...ghtsub=Sortino

Otherwise this question would really require looking at the values seen in those comparisons to see why the sortino had been calculated to that value. Have you tried to extract the values used in each test and manually calculate it based on the help guide description of the equation?

I look forward to being of further assistance

[ozziyhtomit]

Hi,

Thanks for the reply. Yes you guys share the Sortino calc assumptions in that link, thank you for that. That has allowed me to match my calc to yours.

Even knowing that, it does not explain away what I am seeing when I compare the Walk Forward Optimization summary Sortino to the Backtest summary Sortino. I manipulated the parameter values such that the two should generate identical results and the are not close.

Can we schedule a time to share my screen? I need to understand this moving forward!

Thank you,
Tim

[NinjaTrader_Jesse]

Hello ozziyhtomit,

We generally can't provide calls for scripting concerns, if you have a specific situation and sample script that can replicate that result you could attach that and we can review that situation. Alternatively if you can reproduce this with the SampleMACrossOver and provide steps to see that using the sample I could research that further using the sample.

Please let me know if I may be of additional assistance.

[ozziyhtomit]

OK thanks for your help Jesse