Walk Forward Optmization and weekends

[Strategery]

Thanks, Brandon

Please share the entire thread, as my last post is an example of a test being executed when there were no historical segments of market data at all for optimization.
That is obviously an issue but it all stems from the same underlying problem. The module should only optimize or test as the documentation states: "on a historical segment of market data" not a historical segment of time.
That means a trading session.
Regarding partial data, like when there is only one trading session in a 4-day optimization over a 3-day weekend:
A strategy that measures volatility or pretty much any metric over 4 days can not run properly on one or two, or three days' worth of data.
If the three sessions prior to the only one that was used for optimization were very volatile and the fourth wasn't the strategy has no idea that the market was extremely volatile only two trading sessions ago. The optimization pretty much clones the one low volatility day and applies it over the 3 day weekend.
On a three-day weekend with the Monday off using a four-day look back, at no point in the following week will the optimization be performed on the needed four sessions of trading.
Tuesday uses 1, Wednesday uses 1, Thursday 2, and Friday 3.
On a regular week, the only day the strategy has the correct amount of data to analyze to make proper forecasts is Friday.
So given that there are holidays, the strategy will be trading using the required amount of market data to analyze less than 20% of the time.

[NinjaTrader_BrandonH]

Hello Strategery,

Thanks for your notes.

Using the example in post # 14 on this forum thread, the data used for the first optimization period would be from Sunday 5/21 after the session opens to the end of the session on Monday 5/22.

Then, the start of the session on Monday 5/22 to the end of the session on Tuesday 5/23 is used for the test period.

See the 'Understanding Walk Forward Results' section of this help guide page:

https://ninjatrader.com/support/helpGuides/nt8/walk_forward_optimize_a_strate.htm

Please let us know if we may assist further.

[Strategery]

I typed the year incorrectly in post 14 when describing the dates. The example in the screenshot and earlier screenshots are from dates in 2022 and are what I am referencing.
As I mentioned in the post, May 23rd falls on a Monday in 2022.
Here is a corrected post:


Let's zoom in on one issue:
Attached is a screenshot of a walk-forward optimization.
Optimize on two days, test on one
Start Date= Saturday 5/21/22 End Date= Wednesday 5/25/22
The Monday 5/23 test is done using parameters 9/50.
Which "historical segments of market data" were used to generate those parameters?

Hello Strategery,

Thanks for your notes.

Using the example in post # 14 on this forum thread, the data used for the first optimization period would be from Sunday 5/21 after the session opens to the end of the session on Monday 5/22.

Then, the start of the session on Monday 5/22 to the end of the session on Tuesday 5/23 is used for the test period.

See the 'Understanding Walk Forward Results' section of this help guide page:
https://ninjatrader.com/support/help...e_a_strate.htm

Please let us know if we may assist further.

[NinjaTrader_BrandonH]

Hello Strategery,

Thanks for your notes.

Since Saturday is not a trading day, there will be no data to test over on that day.

The data used for the first optimization period would be from the session open on Sunday (5/22/2022) to the end of the session on Monday (5/23/2022).

The start of the session on Monday (5/23/2022) to the end of the session on Tuesday (5/24/2022) is used for the test period.

[Strategery]

The trading date of the 24th in the log checks out for me somewhat. I ran a regular optimization on the 23rd and got levels of 15/50 and confirmed the +$599 P&L with a standard backtest.
A lookback setting of two historical segments of market data should not only use one segment of market data on this day just because there was a weekend.
"Historical segments of market data" is the term Ninjatrader uses when describing walk-forward optimization, and how it works in general practice. The algorithm should use the same amount of data segments for every test. A walk-forward optimization is not consistent otherwise. The test done on Tuesday should use parameters from Monday and Friday.

That being said, my question was not regarding the trading date of the 24th.
Please read my question again:
The Monday 5/23 test is done using parameters 9/50.
Which "historical segments of market data" were used to generate those parameters?

[NinjaTrader_BrandonH]

Hello Strategery,

Thanks for your notes.

The Walk Forward Optimization feature in the Strategy Analyzer was designed to generate results based on calendar days, not trading days.

This means that if the Start Date is set to Saturday and the End Date is set to Wednesday, the two days used to generate Monday's results would be Saturday-Sunday and Sunday-Monday.

I understand that you would like the Walk Forward Optimization to be changed so that it only uses trading days to generate results instead of calendar days.

The Development team is tracking interest for this in an internal feature request and your vote have been added to the feature request. This request is being tracked under the number SFT-5949.

I have also added your notes to the feature request so the Development team can take them into consideration.

As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted on the Release Notes page of the Help Guide.

Release Notes — https://ninjatrader.com/support/help...ease_notes.htm

[Strategery]

How can it use Sunday-Monday data to create parameters to test on Monday when the Monday session is Sunday-Monday?
Based on what you have said so far I would assume Monday would be tested using Friday 5:00 PM to Saturday 4:00 PM and Saturday 5:00 PM to Sunday 4:00 PM.
Which as we know, contains no sessions nop data.

[NinjaTrader_BrandonH]

Hello Strategery,

Thanks for your note.

As previously stated the data from Sunday's session open to Monday's session end would be processed for the optimization period used to calculate the Monday results. Since the Start date is set to Saturday and it processes calendar days, Saturday - Sunday would be processed (which contains no session data), and then Sunday - Monday.

If the Start Date is set to Saturday and End Date is set to Wednesday, the calendar days of Saturday-Wednesday would be processed, not Friday. Walk Forward Optimization can only work with the data loaded in the Start Date and End Date fields. Further, Walk Forward Optimization uses calendar days, NOT trading days. This means Saturday would still be processed when the optimization is run even though there is no session data.

The SFT that I added your vote to is tracking interest for having Walk Forward Optimizations only process trading days instead of calendar days. If this feature is implemented, this means Friday and Sunday would be used to calculate values on Monday since Friday and Monday would be trading days and Saturday would be ignored since it is not a trading day.

[Strategery]

The day in the log with a start and end date of Monday 5/23. It shows parameters of 9/50 and a P&L of -$587.
The trading session for Monday 5/23 begins Sunday 5/22 at 5:00 PM and ends Monday 5/23 at 4:00 PM.
I understand the start and end date in the log for a given day to represent the day the test was done and the parameters on that row are the optimized parameters from the previous two(ish) days.
I am asking where the 9/50 came from that created the loss of $587 since the two previous days have no data.
Your reply said:
"As previously stated the data from Sunday's session open to Monday's session end would be processed for the optimization period used to calculate the Monday results. Since the Start date is set to Saturday and it processes calendar days, Saturday - Sunday would be processed (which contains no session data), and then Sunday - Monday."
That speaks to the parameters that would be used on Tuesday's test, no?
I am referring to Monday's test. We know that Tuesday uses only Monday. We covered that in the first section of my last comment.
It reads like you are saying Monday's test is calculated using Monday's data.
This would be impossible without a Delorian and I doubt it would have generated a negative P&L as they would have been optimal parameters.

[NinjaTrader_BrandonH]

Hello Strategery,

Thanks for your note.

I have modified the SampleMACrossover by adding prints of the first bar and last bar used for the optimization/backtest as well as the Fast parameter, Slow parameter, and the optimization instance ID.

When the Walk Forward Optimization is run, optimizations are performed based on the Start date/End date calendar days provided to test each Fast/Slow parameter combination based on the Fast/Slow parameter settings in the Strategy Analyzer.

Then backtests are run after the optimizations and the best Fast/Slow parameter combinations are calculated and printed to the output window. The prints show that the Fast/Slow parameters calculated for Monday's date use historical data from 5/22 to 5/23. And yes, Monday's Fast/Slow parameter combination could calculate as a negative PnL.

See this demonstration video: https://brandonh-ninjatrader.tinytak...Ml8yMTY1MzgwNA

The feature request I added your vote to would allow you to have the WF Optimization calculate based on trading days instead of how it currently uses calendar days. We do not have an ETA for when this feature might be implemented.