Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Market Replay vs Back Test results

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • cfp462
    replied
    Ok so this is starting to make a little more sense, but please note that all my orders are market orders without stops or targets, and calculated on bar close, so the granularity shouldn't really affect my trades.

    And to address the issue with Kagi, how do you explain it have exactly the same orders except inverse in back testing versus replay? From my understanding it is forming the bars at the end of the base period. I guess my question is, how can I get these to line up? How can I instruct my strategy to trade live, exactly how it does in back testing?

    This is kind of a side note but to your point about charts loading at different times. How do I get my Automated Strategies to load prior data after I enable them so it doesn't take days for them to start trading?

    Leave a comment:


  • NinjaTrader_AdamP
    replied
    cfp,

    Likely you are noticing two separate issues here. One is order management having granularity in Market Replay, but not having it in a backtest. The second is probably that Kagi chart styles are "time-independent", so there are differences on your charts.

    In Market-Replay, order management can occur intra-bar as it would on a real-time connection. As such stops and targets could be filled intra-bar even if the strategy is operating tick-by-tick. This is because the backtesting engine lacks intra-bar granularity to determine how a bar was built internally. It only has OHLC data available.

    You can use this reference sample to backtest tick-by-tick : http://www.ninjatrader.com/support/f...ead.php?t=6652

    Kagi bar styles are "time independent".

    “Time independence” implies that these bars are not fixed to a set time scale, so for example, a bar “closes” when some other condition not related to time is reached, such as a certain number of ticks occurred, or a certain price range was achieved. Bars that fall into this category are range, Renko, tick, volume, etc.

    Time-independent bar styles are mostly built from information received from tick-data, and each bar’s time stamp is only dependent on when the condition for these bars to “close” was reached. As such one bar may take 1 minute 32 seconds to “close”, the next may take 1 minute 14 seconds to “close”.

    As a side effect, when reloading this style of chart you may notice differences in the way the bars plot as “time independent” charts are sensitive to their initial starting point. I.e. there may be some missing ticks that would have built the first bar on your chart differently and it cascades through the whole chart.

    Leave a comment:


  • cfp462
    replied
    Hey, I am also having pretty significant problems with back testing versus market replay. I have both a Zen Fire connection and CQG, and my problem seems to exist regardless of what data I use.

    The issue is that back testing is not even remotely close to Market Replay. Which doesn't make sense because even if I use ZenFire tick data, calculating on bar close = true, my results can still be significantly different then market replay.

    Issues I have encountered include (but are not limited to):

    Market replay can make almost exactly the opposite trades as back testing

    Time of entry and exit of trades are different by the time of base period value

    When exit on close = true backtesting will make trades that enter and exit on the same day at exactly the same time and record a significant profit

    When back testing a kagi 10 minute base period value versus a kagi 600 second base period value with a MACrossOver strategy CalculateOnBareClose = true; will result in very different results

    I have been testing the NT platform for the past month and cannot seem to understand all of these problems. I understand that data will be slightly different but I do not understand why I am having problems quite like this.

    Thanks for any help that can be provided

    Leave a comment:


  • NinjaTrader_Jason
    replied
    Yes, you could say that because the strategy will be calculate intrabar (providing Calculate on bar close = False). Upon backtesting, the strategy cannot be calculated intrabar - only at the end of each price bar.

    Yes, it will be close - the strategy will be calculated as per the same real-time data.

    Leave a comment:


  • GKonheiser
    replied
    Thanks Jason,

    Would you say that market replay is therefor a more accurate representation of live trading then back testing?

    If I were to use the same data source for live trading as the market replay source uses should I expect a very close coloration between market replay and live trading results or would there still be significant errors?

    Leave a comment:


  • NinjaTrader_Jason
    replied
    Hello Gkonheiser,

    Market replay data you download is recorded from real-time Zen-Fire data. So it is a different data source than Kinetick and can cause subtle differences.

    Please note that differences between backtests and real-time tests (this includes the Market Replay) is expected. See the link below for more information.

    Leave a comment:


  • GKonheiser
    started a topic Market Replay vs Back Test results

    Market Replay vs Back Test results

    I am noticing differences in System performance between back testing and Market replay and I can see its due to differences in the data. Is the market replay tick data unfiltered, as I'm pretty sure my historical data is from Kinetick which is also unfiltered so in theory there should be no difference?

Latest Posts

Collapse

Topics Statistics Last Post
Started by JackyP, 07-25-2023, 08:39 AM
3 responses
312 views
0 likes
Last Post Lancer
by Lancer
 
Started by TiffanyMiller, Yesterday, 10:19 PM
0 responses
2 views
0 likes
Last Post TiffanyMiller  
Started by algospoke, 05-13-2024, 06:53 PM
12 responses
133 views
0 likes
Last Post algospoke  
Started by psangram, Yesterday, 06:35 PM
0 responses
4 views
0 likes
Last Post psangram  
Started by Alaina19, 07-14-2024, 04:38 PM
2 responses
16 views
0 likes
Last Post Alaina19  
Working...
X