Backtesting Custom Strategy and timezone issues

[mgalal]

Hi, I have a custom code strategy that has such variables as trading sessions (Start/exit), trade profit target/loss, daily profit target/loss, and weekly profit target/loss. I ran into issues when the trading session is setup to be between 8PM and 9:30AM EST. In backtests, it works great and as expected. In live trading, there an issue with date change around mid-night that causes the strategy to trade again even if daily profit target is reached, which is not my intent. For example, If I am running the same strategy for 12 tick target on /ES starting at 8PM, I expect it to stop trading if target is reached before 12:00am and until 9:30 (session end). In reality, it will take the profit target and after midnight, the daily profit target logic kicks in assuming a new day and trade again.

The coding aspect is very complex to check if target is reach in the current session only, not the previous day, which would defeat the purpose of daily target.. anyway, an easy workaround for me was to set the NT8 Clock to UTC and set the trading session start at 12:00AM, which is 8PM previous date. It works great. Now all of this is context..

Here's the real problem:
- When using UTC in Strategy Analyzer, setting the strategy start time/endtime to UTC produces very different results than all prior backtests for the same time period.
- Using EST settings for the 8PM-9:30am produces the same results seen both in previous backtests and live tradings.
- It is as-if NT calculates the UTC and EST differently in real-time vs historical data.
- Changing NT/Computer clock back to EST and picking a non-midnight time segment produces consistent results between live/real-time trades.
- Changing NT/Computer clock to UTC, but setting trading session in backtest to EST produces also consistent results from line above and real-time.

For example:
- NT8/Computer in UTC, real-time trading starting 13:30 (09:30am) matches Strategy Analyzer results only if the backtest start time is set to 09:30EST not 13:30 UTC. The strategy doesn't have any hard coded timezone logic. Rather checks if CurrentTime > StrategySessionStart.

I am getting thrown off that offset from UTC and wondering in NT caches historical data differently and doesn't use UTC offset. How do I begin to even troubleshoot this issue?

[NinjaTrader_ChelseaB]

Hello mgalal,

All data in NinjaTrader is converted to the time zone of the computer PC Clock.

If the pc clock is set to Eastern, all real-time data updates in the script, and all saved historical data is saved with the Eastern time zone.

What the pc clock set to on this computer?

The historical and real-time data are not timestamped differently and both will be set to the time zone of the PC.

If the time zone on the PC is changed or the pc clock is synchronized to internet time, the historical data must be re-downloaded.
https://ninjatrader.com/support/foru...059#post846059

That said, when exporting to text file or importing from text file, this will be UTC.


If there is a bug or an issue, print out the information and post the output so that we may see.

Below is a link to a forum post that demonstrates using Print() to understand behavior.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working?p=791121#post791121

[mgalal]

Hi, sorry for the delayed reply. The PC is set to EST, and NT8 is set to UTC. Backtesting doesn't produce different results, but I am suspicious that this causes other unforseen issues.

[NinjaTrader_ChelseaB]

Hello mgalal,

As a heads up, there have been reported issues in the past with specific connection types when changing the time zone of NinjaTrader to not match the PC.

If NinjaTrader is set to the same time zone as the PC, then all historical data is deleted first, and then re-downloaded, can you reproduce?

https://ninjatrader.com/support/helpGuides/nt8/index.html?editing.htm#HowToDeleteHistoricalData

[mgalal]

Thank you. The real issue if crossing the midnight into a new date if I want to trade from 8pm-2pm EST. Things get thrown off after midnight. If there is a way to achieve that when start time is 8pm (on 6/21) to 2pm (on 6/22). Any suggestions would be appreciated.

[NinjaTrader_ChelseaB]

Hello mgalal,

This is being reproduced with the time zone set to the pc time zone and all historical data reloaded?

[mgalal]

I am not sure how to clear that data up. Are you speaking of market data download for Playback?

This issue is that I want to trade starting at 8PM ending at 4PM. UTC gives me a nice way to trade starting at midnight for the same effect I want. The problem is the PC time always switches back to actual cloud zone and at some point, NT switches from UTC to EST. Sometimes that will happen in a day or 2 or sometimes a full week. Ideally, I can keep it EST and just be able to set the right conditions to trade between 8PM and 4PM. If StartTime=8PM, and EndTime=4PM, Strategy always assume 4PM comes before 8PM. Perhaps there is a tick to check the date as well along with time?

[NinjaTrader_ChelseaB]

Hello mgalal,

Historical data would be downloaded from the data provider. Market Replay data would be downloaded from NinjaTrader servers.

You can simply delete the historical data and market replay data.

https://ninjatrader.com/support/helpGuides/nt8/index.html?editing.htm#HowToDeleteHistoricalData

I am asking you to change the time zone of NinjaTrader to match the time zone of the PC and then delete historical and market replay data, and then tell me if you can reproduce.

[mgalal]

Hi Chelsea, Just to be clear, are you saying I should delete the data first and then confirm if the 8PM-4PM in strategy is still and issue? I can try that. Are there specific instructions to make sure both historical data and market download data is deleted? Is it just going to Historical data and right click "remove"?

[mgalal]

Hi Chelsea, that didn't work. changed timezone to EST, 8pm-4pm produces no trades, simply changing it to 8pm-11:59PM produces trades and shows historical trades.

[NinjaTrader_ChelseaB]

Hello mgalal,

Thank you for confirming you have deleted all historical data, and that the behavior persists when using Eastern time with newly downloaded historical data.

This would indicate an issue with the logic in the script and not with the time zone, as the behavior can be reproduced using Eastern time.

To understand why the script is behaving as it is, such as placing orders or not placing orders or drawing objects when expected, it is necessary to add prints to the script that print the values used for the logic of the script to understand how the script is evaluating.

In the strategy add prints (outside of any conditions) that print the values of every variable used in every condition that places an order along with the time of that bar.

Prints will appear in the NinjaScript Output window (New > NinjaScript Output window).

Also, enable TraceOrders which will let us know if any orders are being ignored and not being submitted when the condition to place the orders is evaluating as true.

I am happy to assist you with analyzing the output from the output window.

Run or backtest the script and when the output from the output window appears save this by right-clicking the output window and selecting Save As... -> give the output file a name and save -> then attach the output to your reply.

Below is a link to a forum post that demonstrates using prints to understand behavior and includes a link to a video recorded using the Strategy Builder to add prints.
https://ninjatrader.com/support/foru...121#post791121

Please let me know if I may further assist with analyzing the output or if you need any assistance creating a print or enabling TraceOrders.

[mgalal]

Chelsea, thanks for the reply. The issue is much simpler than this. When you evaluate if current bar is more than start time or less than end time, the problem is that 8PM is > 4PM in comparison, and doesn't take into account that you're trading overnight, so conditions don't evaluate correctly. I think I have figured out a solution and I am testing it. Once I confirm it works reasonable well. I'll post the sample strategy to the community. I just have lots of extra conditions to check if it's an evening to next day trade, before midnight, after midnight, etc.

[NinjaTrader_ChelseaB]

Hello mgalal,

If you are using time conditions in the logic of the script, this would be affected by the time zone.

Printing all values in all conditions would tell you exactly when the conditions are true.

[mgalal]

Chelsea, I'll do that. Is printing better performing than Logging? Would it only affect resources when Output window is used? I am pretty close, but something is off with my exit conditions.. still trades bit after exit conditions, but with least frequency even going a full 6 months range..

[mgalal]

Chelsea, I found the issue. It was a simple user error about "if all" vs "if any", that addressed it. However, in strategy builder, comparing time is of the current bar[0] <= end time. Is there a way to get current timestamp in strategy builder to exit at an exact time instead of waiting until next bar forms to get its time?