I've been backtesting a strategy with the Market Replay Connection for approximately 3 weeks. I've noticed that the same strategy may give different results. So I'm concerned that I'm not using Ninjatrader properly to run iterative script changes.
Here is how I've been iteratively testing changes to my script:
- Run the strategy
- Like the results, so save the .cs file in another directory (as a backup --- e.g., copy and paste the .cs file in an alternate file system folder).
- Make changes to the strategy from within Ninjatrader (using Tools -> Edit Ninjascript -> Strategy...)
- Recompile the script (with F5), and rewind the market replay connection
- Remove and re-add the strategies on the Strategies tab
- Rerun the strategy
- Don't like the results. So close the strategy editor in Ninjatrader, and copy over the new .cs file with the backup .cs file directly in the filesystem folder.
- Reopen the strategy editor to confirm that the new strategy is loaded. Type a period in the script and delete it, and then recompile the strategy to make sure that it's loaded.
- Remove and re-add the strategies on the Strategies tab. Also, rewind the market replay connection
- Rerun the strategy
My results differ -- which I expect. But sometimes, it's significant ($100+). Should that be expected?
Also, if I delete the cache under Ninjatrader 7\db\cache, restart Ninjatrader, and rerun; the results are also significantly different (in the order of several $100's). In this scenario, some instruments don't even have trades executed when previously, trades were executed frequently. If I rewind the market replay connection and rerun the strategy, the trades fire. But what just happened, lol?
What is the right way to run iterative backtests while tweaking a strategy file? Should I delete the cache and restart Ninjatrader everytime? I'm very concerned that the results aren't mimicking what I'd get in real time. I'm running NT 64-bit 7.0.1000.27, and using backtesting data downloaded from marketreplay.net and marketreplaydata.com. I'm using Forex pairs.
I have the simulator set to enforce immediate fills, and to immediately submit live historical orders. I always replay the market replay connection from the same day/time.
If you need anymore information, please let me know. Thanks for your feedback.
Here is how I've been iteratively testing changes to my script:
- Run the strategy
- Like the results, so save the .cs file in another directory (as a backup --- e.g., copy and paste the .cs file in an alternate file system folder).
- Make changes to the strategy from within Ninjatrader (using Tools -> Edit Ninjascript -> Strategy...)
- Recompile the script (with F5), and rewind the market replay connection
- Remove and re-add the strategies on the Strategies tab
- Rerun the strategy
- Don't like the results. So close the strategy editor in Ninjatrader, and copy over the new .cs file with the backup .cs file directly in the filesystem folder.
- Reopen the strategy editor to confirm that the new strategy is loaded. Type a period in the script and delete it, and then recompile the strategy to make sure that it's loaded.
- Remove and re-add the strategies on the Strategies tab. Also, rewind the market replay connection
- Rerun the strategy
My results differ -- which I expect. But sometimes, it's significant ($100+). Should that be expected?
Also, if I delete the cache under Ninjatrader 7\db\cache, restart Ninjatrader, and rerun; the results are also significantly different (in the order of several $100's). In this scenario, some instruments don't even have trades executed when previously, trades were executed frequently. If I rewind the market replay connection and rerun the strategy, the trades fire. But what just happened, lol?
What is the right way to run iterative backtests while tweaking a strategy file? Should I delete the cache and restart Ninjatrader everytime? I'm very concerned that the results aren't mimicking what I'd get in real time. I'm running NT 64-bit 7.0.1000.27, and using backtesting data downloaded from marketreplay.net and marketreplaydata.com. I'm using Forex pairs.
I have the simulator set to enforce immediate fills, and to immediately submit live historical orders. I always replay the market replay connection from the same day/time.
If you need anymore information, please let me know. Thanks for your feedback.
Comment