Optimization result not reflected in panel

[cmarkb]

Hi. We're trying out Optimization in the Strategy Analyzer. The results of the Optimizer run though isn't shown in the upper panel. Display:Summary panel though shows result of the run.
What could be the cause for this?
Thanks.

[NinjaTrader_Kate]

Hello cmarkb,

Thank you for your post.

What version of the platform are you running? You can find this under Help > About. The current version is 8.0.24.3.

If you're on a different version, please update the platform and run another optimization test. To update NinjaTrader, please follow the steps below:

• First, copy your license key from NinjaTrader under Help> License Key then exit NinjaTrader
• Click on the link: https://ninjatrader.com/PlatformDirect
• Enter your license key and press Submit
• Select 'Download'
• Critical: Before running the installer, ensure NinjaTrader is closed.

If you're already on the current version or updating did not resolve it, please try temporarily renaming the database to see if this resolves the issue.

• Shutdown NinjaTrader and Go to the Start Menu
• Select My Documents--> NinjaTrader 8--> db--> "NinjaTrader.sdf" file (may only say 'NinjaTrader' depending on your Windows File Explorer settings)
• Right click on NinjaTrader.sdf and select "Rename" *Name it "OLDNinjaTrader.sdf"
• Then restart the software and NinjaTrader will create a fresh database file to use
• Restart NinjaTrader and test again.

If this resolves the issue, you'll need to keep the new database.

• Unfortunately the following items stored in the old database will be lost - Trading Hours templates, Instrument Lists / Custom Instruments, historical trade execution data, saved Strategy Analyzer backtest results

If it does not, you can go back to the db folder, delete the NinjaTrader.sdf file and rename the OLDNinjaTrader.sdf file back to just NInjaTrader.sdf to restore your original database. Let me know if this is the case and we can look further into the issue.

Thanks in advance; I look forward to assisting you further.

[cmarkb]

Okay. Seems like the answer from other thread worked (https://ninjatrader.com/support/foru...e-best-results).
Removing a reference to Instrument that's inside State == State.Configure code (and optionally also setting BarsRequiredToTrade to 14 -- this is from another thread) made it display the results.

[cmarkb]

Okay it's not yet completely resolved. The results are all the same. And the result doesn't match one that's displayed below in the Display "Summary" panel.

[cmarkb]

Hi Kate,

Okay will try that tomorrow and post here the results.

[cmarkb]

Hi Kate,

I've tried updating to the latest NT and renaming NinjaTrader.sdf to let NT generate a new one but the issue still persist. Top panel has same results on all iteration (value is different from previous screenshot).

(After update to latest NT)



(After generating new NinjaTrader.sdf)

[NinjaTrader_Kate]

Hello cmarkb,

Thank you for your reply.

Let's make sure your data isn't caching something and causing strange results.

To delete the cache, follow the steps below.

• Shut down NinjaTrader.
• Open the Documents > NinjaTrader 8 > db folder.
• Delete the sub-folder named 'cache'.
• Restart NinjaTrader and test.

Please let us know if we may be of further assistance.

[cmarkb]

Hi Kate,

Thanks. Okay I've removed the cache directory and reran it but the issue is still there.

[NinjaTrader_Kate]

Hello cmarkb,

Thank you for your reply.

Do you see the same when testing with the built in Sample MA Crossover?

Thanks in advance; I look forward to assisting you further.

[cmarkb]

Hi Kate,

Thanks. I've tested with Sample MA CrossOver and it works for that one. I've also tried recreating the issue on the Sample MA CrossOver by copying how built-in properties where set in our backstrategy (and also adding additional data series) and I wasn't able to recreate the issue.

(Sample MA Crossover result)


Our backstrategy does use our own indicator which reads in data from a file. Will try recreating that in the sample ma crossover and see if it does something with the issue.

But please keep suggesting possible fixes. Thanks.

[NinjaTrader_Kate]

Hello cmarkb,

Thank you for your reply.

In consulting with my team they've postulated that it may be something in OnStateChange() causing the issue. Could you provide a sample of your OnStateChange() section of your code?

If you'd rather not post that publicly, you can email it to us at scriptingsupport [at] ninjatrader [dot] com with 3164397 ATTN Kate W. in the subject line and a link to this post in the body of the email.

Thanks in advance; I look forward to assisting you further.

[cmarkb]

Hi Kate,

Thanks. Done sending email attached with contents of our OnstateChange().

I've also tested something else. So our indicator serves as source of entry signal. It reads values from a file and matches it to the tick timestamp.
We also have BBands as another entry signal source. So I added that back in, enabled BBands as signal, and while Optimizer was running I can see in the NinjaScript Output window that our indicator was also working reading in the file although its values won't be used as entry signal.
When the Optimizer was done running it was able to display different results for each iteration on the top panel.

So from this discovery I thought maybe it's the location where I read in the value from our indicator that was causing the issue somehow so I moved it up a location in the OnBarUpdate() (somewhere it's processing non-Tick data series) but that didn't work.

(BBands As Signal)

[NinjaTrader_Kate]

Hello cmarkb,

Thank you for your reply.

Looking at your OnStateChange we see a few things that are concerning - using async on NinjaTrader methods is not recommended, using random with the optimizer is also a no-no. That's not specifically in your code but one of my colleagues noted there's nods to it in what you did provide. Also, it looks like you're referencing some ATM strategy items, which we'd expect to not be backtestable.

What we'd recommend is implementing functionality in pieces until the issue comes up, then comment out the most recently added functionality and test to confirm which specific part is causing the issue. I'd start with ruling out the few items I mentioned above, possibly by adding functionality to the Sample MA Crossover until it breaks. From there, we can be more helpful if you haven't already gotten to the bottom of it.

Thanks in advance; I look forward to assisting you further.

[cmarkb]

Hi Kate,

Thanks. Will try out your suggestion.

[cmarkb]

Hi Kate,

Okay turns out the one causing the issue was one of our custom parameter. By default it just displays a readonly string of values. Clicking on a button pops out a form where you set the values. Then you pass this custom param to the indicator.
Somehow on each iteration of the optimizer the custom parameter has the same value. That's why the result has the same value because same param was provided.
Thanks for the help.