Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

backtest replication

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    backtest replication

    I back tested strategy with Matlab. The result is happy one. I did implement it in (free) NT. The result is very close to one I had in MatLab which makes me confident of analysis.

    However, the test with real-time feed shows Veeeery different result... I could agree about different execution price, but not with large time delay. NT calculates triggers at different bars.

    I saw disclaimer of NT about possible backtest-execution difference, which makes me uneasy since I am used to such difference of <1-3%, which is ONLY due to price, but almost never due to timing. With NT this difference is huge (can be few times).

    Could you share your experience and tricks?

    I suspect something has to do with updateOnBar option - end of bar or continuous. What is the difference? How update is done? I tried both with almost equaly bad result.

    Thanks
    Nik

    #2
    Hi Nik, welcome to our support forums! Which market are you using? There are two settings, one updates the calculations on every tick (CalculateOnBarClose = false), the other at the end of the bar (set the previous to true) - http://www.ninjatrader-support.com/H...BarClose1.html

    For finer backtesting you might want to check into this concept - http://www.ninjatrader-support2.com/...ead.php?t=6652
    BertrandNinjaTrader Customer Service

    Comment


      #3
      I use Eironext (cash market) + IB as data-feed and execution platform.

      I think Euronext (or anything else) is irrelevant as the test is done with free version where only "fake" trades are booked within NT. Also I said initially that with my other strategy which is "tick-by-tick" I managed to replicate my end-of-day real PnL with backtest to satisfactory level.

      That system sends Market orders immediately after the trigger occurs. In case of NT I expect that orders will be issued immediately after trigger at the end-of-bar. The problem is that these triggers do not coincide in case of real-time and backtest.

      Another, related question - how do you store indicator bar series when indicator values are updated tick-by-tick?

      I am considering two possibilities for CalculateOnBarClose = false:
      - IndicatorSeries[0] is updated within the current bar, while [1],[2]... are fixed because past bars do not change anymore. Once current bar closes IndicatorSeries shifts;
      - when IndicatorSeries[0] is updated whole series shifts, therefore [1] value relates to the previous tick, not bar.

      which one is used?

      when CalculateOnBarClose = true, IndicatorSeries[0][1][2]... must contain info related to bars and must be identical to my backtest results (I leave out the execution process).
      Last edited by nikkk; 04-19-2009, 06:30 AM.

      Comment


        #4
        With CalculateOnBarClose set to true the orders will be send after the bar is closed, that means on the open tick of the next bar.

        This option will only affect accessing the last bar on the chart in realtime, with this set to false this bar and the calculations in the OnBarUpdate() will update on every incoming tick. If the bar closes, the values become 'permanent' and the cycle starts all over for the next bar to be completed.
        BertrandNinjaTrader Customer Service

        Comment


          #5
          Seems I have to debug more. Therefore two more questions:
          1. can I output Print() into Log file?
          2. do you have flag which tells me that the Bar starts (or ends) during real-time?

          Regards,
          Nik

          Comment


            #6
            Hi, no but you can copy and paste with the normal windows commands from the output window. For realtime order debugging setting TraceOrders to true in the Initialize() will be helpful, for the flag you can work with FirstTickOfBar - http://www.ninjatrader-support.com/H...TickOfBar.html
            BertrandNinjaTrader Customer Service

            Comment


              #7
              is there any way to see the Caller? I am interested in the flag which tells me if my strategy is called either from backtest (optimizer) or from real-time stream.

              thanks
              Nik

              Comment


                #8
                Hi Nik, you could work with Historical for this - http://www.ninjatrader-support.com/H...istorical.html
                BertrandNinjaTrader Customer Service

                Comment


                  #9
                  I stored today quotes and replay those through "Market Replay connection".

                  Strange thing I observe is that if I start replay at let say 8.50, indicators values will get different from if I would start replay at 9.03... How to get my calculations replicated...

                  What is the size of buffer in outout window?

                  I still believe that having file-output is crucial for automated post-trade analysis and cost control. Manual copying is rudimentary...

                  Also I noticed that if I Print detailed output it loads memory and slows computer. This problem would be solved with log-file. Dont understand why users cannot have it. It's trivial.

                  Comment


                    #10
                    If you want to output to file you need to do it in your code. http://www.ninjatrader-support2.com/...ead.php?t=3475

                    I do not know the exact size of the Output Window buffer.

                    Please try reloading your NinjaScript as you load it up at different times in your replay. They should generate the same values.
                    Josh P.NinjaTrader Customer Service

                    Comment


                      #11
                      Thanks a lot for all your useful replies. Still a question I have.

                      Now, for two conseq. days my strategy is run during a day with life-feed (A). I get result based on CalculateOnBarClose=false.

                      When I replay the same day, I get different result (B). I do not apply "bad tick" filter, which means I take-in everything (right?).

                      Next to this I replay same strategy with CalculateOnBarClose=true (C). In addition, I run backtest (D).

                      All four numbers (A,B,C,D) are different and for two days I cannot say they even close:

                      (each row reps 2 days)
                      A = 1,360 1640
                      B = 170 640

                      C = (87) 320
                      D = (140) (220)


                      I would accept the difference of <100.

                      The algorithm is briefly the following:

                      if (prefilter
                      && sessionTime()
                      && BarsSinceExit() !=0) {
                      if (buy-condition && Position!=Long) {
                      EnterLong()
                      }
                      if (sell-condition && Position!=Short) {
                      EnterShort()
                      }
                      }

                      if(BarsSinceEntry()>=1) {
                      if(stop-signal && position==Long) ExitLong()
                      if(stop-signal && position==Short) ExitShort()
                      }

                      if (closeTime()) {
                      if(position==Long) ExitLong()
                      if(position==Short) ExitShort()
                      }
                      Last edited by nikkk; 04-22-2009, 03:52 PM.

                      Comment


                        #12
                        What kind of chart are you running? Tick? Minute?
                        Josh P.NinjaTrader Customer Service

                        Comment


                          #13
                          the chart is 1 minute. the strategy is designed to run on bar data.
                          i presume, in the replay mode the result should be equivalent to my real-life run.

                          Comment


                            #14
                            Not necessarily. Please go to Tools -> Options -> Simulator tab and ensure you don't have any delays set. These delays do influence your fills drastically from accelerated replay attempts.
                            Josh P.NinjaTrader Customer Service

                            Comment


                              #15
                              Originally posted by NinjaTrader_Josh View Post
                              Not necessarily. Please go to Tools -> Options -> Simulator tab and ensure you don't have any delays set. These delays do influence your fills drastically from accelerated replay attempts.
                              Not according to 6.5.1009 help file.

                              Click image for larger version

Name:	Capture.PNG
Views:	1
Size:	50.7 KB
ID:	852135

                              Mike

                              Comment

                              Latest Posts

                              Collapse

                              Topics Statistics Last Post
                              Started by dtaylor, 08-07-2022, 09:43 PM
                              4 responses
                              578 views
                              0 likes
                              Last Post Emma1
                              by Emma1
                               
                              Started by edward_steve12, Today, 03:23 AM
                              1 response
                              3 views
                              0 likes
                              Last Post Emma1
                              by Emma1
                               
                              Started by joseph_kuruvilla12, Today, 01:01 AM
                              1 response
                              7 views
                              0 likes
                              Last Post Emma1
                              by Emma1
                               
                              Started by samish18, Today, 04:01 AM
                              1 response
                              8 views
                              0 likes
                              Last Post Emma1
                              by Emma1
                               
                              Started by greenberet, Today, 04:05 AM
                              0 responses
                              3 views
                              0 likes
                              Last Post greenberet  
                              Working...
                              X