Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Calculating latency

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Calculating latency

    I live in a remote area of South America and installed Starlink last weekend.

    great bandwidth!

    the latency to the USA is about the same as local Wifi providers

    am trying to determine the order status latency, including the turnaround at the exchange

    here are the trace file entries from CreatOrder through OrderState Working

    2024-03-04 07:46:16:015 (Apex Rithmic) Cbi.Account.CreateOrder: orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Initialized instrument='NQ 03-24' orderAction=Buy orderType='Limit' limitPrice=18351 stopPrice=0 quantity=1 tif=Day oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=94538 time='2024-03-04 07:46:16' gtd='2099-12-01' statementDate='2024-03-04' id=-1 comment=''
    2024-03-04 07:46:16:040 (Apex Rithmic) Cbi.Account.Submit0: realOrderState=Initialized isPendingSubmit=False orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Initialized instrument='NQ 03-24' orderAction=Buy orderType='Limit' limitPrice=18351 stopPrice=0 quantity=1 tif=Day oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=94538 time='2024-03-04 07:46:16' gtd='2099-12-01' statementDate='2024-03-04'
    2024-03-04 07:46:16:040 (Apex Rithmic) Cbi.Account.Submit1: realOrderState=Initialized orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Submitted instrument='NQ 03-24' orderAction=Buy orderType='Limit' limitPrice=18351 stopPrice=0 quantity=1 tif=Day oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=94538 time='2024-03-04 07:46:16' gtd='2099-12-01' statementDate='2024-03-04'
    2024-03-04 07:46:16:040 (Apex Rithmic) Cbi.Simulator.Submit: realOrderState=Initialized orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Submitted instrument='NQ 03-24' orderAction=Buy orderType='Limit' limitPrice=18351 stopPrice=0 quantity=1 tif=Day oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=94538 time='2024-03-04 07:46:16' gtd='2099-12-01' statementDate='2024-03-04' delay=100
    2024-03-04 07:46:16:040 (Apex Rithmic) Cbi.Account.OrderUpdateCallback: realOrderState=Submitted orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Submitted instrument='NQ 03-24' orderAction=Buy limitPrice=18351 stopPrice=0 quantity=1 orderType='Limit' filled=0 averageFillPrice=0 time='2024-03-04 07:46:16' statementDate='2024-03-04' error=NoError comment='' nr=-1
    2024-03-04 07:46:16:204 (Apex Rithmic) Cbi.Account.OrderUpdateCallback: realOrderState=Accepted orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Accepted instrument='NQ 03-24' orderAction=Buy limitPrice=18351 stopPrice=0 quantity=1 orderType='Limit' filled=0 averageFillPrice=0 time='2024-03-04 07:46:16' statementDate='2024-03-04' error=NoError comment='' nr=-1
    2024-03-04 07:46:16:204 (Apex Rithmic) Cbi.Account.OrderUpdateCallback: realOrderState=Working orderId='8b0aba1111764419b14d713d9968ea53' account='Sim101' name='Entry' orderState=Working instrument='NQ 03-24' orderAction=Buy limitPrice=18351 stopPrice=0 quantity=1 orderType='Limit' filled=0 averageFillPrice=0 time='2024-03-04 07:46:16' statementDate='2024-03-04' error=NoError comment='' nr=-1


    The order was Submitted at 07:46:16:040 and was reported Working at 07:46:16:204

    So, I don't remember what the units are for 040 and 204.

    Are those milliseconds?

    What is the latency here?

    204 - 40 = 164 (? units??)



    would this latency be different with the live account?




    #2
    It seems that the last units in the time format are milliseconds

    then the latency here would be 164 milliseconds between order submission and working status (at the workstation)

    other opinion?

    Comment


      #3
      Are you using Sim101?

      It seems so, according to your log.

      In this case, the 'latency' you've calculated
      is not truly representative of any real-world
      trading.

      The only times you're really measuring are
      whatever the Sim101 engine is doing after it
      submits the order and updates the order to a
      working state.

      The Sim101 engine never interacts with anything
      in the outside world (such as the exchange). It
      is completely localized and self contained.

      In fact, the only thing that is affecting the speed
      of the state transition you've measured is probably
      your CPU -- you're certainly not measuring any
      kind of network-based speed -- the Sim101
      engine does not need the network for that order
      state transition.

      ​You'd probably get the same times using
      Market Replay, I suspect, since the Playback
      connection is a local data feed being used to
      supply past historical data to the same local
      Sim101 engine -- no network is involved.

      In your case, since you are connected to a
      Rithmic account, the only thing the network
      is being used for is a live data feed.

      Just my 2˘.

      Last edited by bltdavid; 03-04-2024, 10:08 AM.

      Comment


        #4
        Ok, will redo it with the live account

        (thought the SIM101 engine was not local)

        Comment


          #5
          Originally posted by llanqui View Post
          Ok, will redo it with the live account

          (thought the SIM101 engine was not local)
          Yes, that would be much better.

          It's still useful to know the value you calculated from
          the Sim101 trade -- that calculated value represents
          an absolute best case scenario time for a real-world
          trading account -- a time you will surely never see.

          So, when you get your live account times, they will
          obviously be much worse than 164 ms.

          That value is still useful in a further calculation.

          How so?

          Let's say you calculate a value 800 ms for the order
          transition in your real world trading account.

          Well, now I'd suggest you subtract the 164 ms from
          the network-based 800 ms (because 164 represents
          internal work done for a non-network order transition),
          so by subtracting out that internal time, it's a good
          assumption that the remaining 636 ms is now truly
          representative of the network overhead of the order
          state transition.

          Make sense?

          Comment


            #6
            with the live account

            2024-03-04 11:48:29:582 (Apex Rithmic) Rithmic.Adapter.Submit1: orderId='b92d356908554e6f947e56f3c21b78ec' account='PA-APEX-xxxxxxxx' name='Entry' orderState=Initialized instrument='NQ 03-24' orderAction=Sell orderType='Stop Limit' limitPrice=18276.75 stopPrice=18276.75 quantity=10 tif=Day oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=94634 time='2024-03-04 11:48:29' gtd='2099-12-01' statementDate='2024-03-04'
            2024-03-04 11:48:29:645 (Apex Rithmic) Cbi.Account.OrderUpdateCallback: realOrderState=Submitted orderId='b92d356908554e6f947e56f3c21b78ec' account='PA-APEX-xxxxxxxx' name='Entry' orderState=Submitted instrument='NQ 03-24' orderAction=Sell limitPrice=18276.75 stopPrice=18276.75 quantity=10 orderType='Stop Limit' filled=0 averageFillPrice=0 time='2024-03-04 11:48:29' statementDate='2024-03-04' error=NoError comment='' nr=-1
            2024-03-04 11:48:29:850 (Apex Rithmic) Cbi.Account.OrderUpdateCallback: realOrderState=Accepted oldOrderId='b92d356908554e6f947e56f3c21b78ec' orderId='1107667168' account='PA-APEX-xxxxxxxx' name='Entry' orderState=Accepted instrument='NQ 03-24' orderAction=Sell limitPrice=18276.75 stopPrice=18276.75 quantity=10 orderType='Stop Limit' filled=0 averageFillPrice=0 time='2024-03-04 11:48:30' statementDate='2024-03-04' error=NoError comment='' nr=-1


            11:48:29:582 submitted by Rithmic Adapter at Ninja WorkStation
            11:48:29:645 exchange acknowledges Submission (received at Ninja WorkStation)
            11:48:29:850 exchange notifies Order is in Working Status (received at Ninja WorkStation)


            Comment


              #7
              And, because you're using an Apex/Rithmic account,
              you have to consider whether you're using an evaluation
              account or a performance account for your measurements.

              Why?

              Because orders for eval accts never go to the exchange,
              they are simulated on Apex's servers (er, I mean the
              Rithmic servers that Apex is leasing) -- remember, the
              eval acct is being simulated remotely.

              The Sim101 account is simulated locally.

              But, did you know your performance acct could also be
              simulated remotely?

              Yep.

              Apex might be doing that, esp for newer perf accts,
              these go through another perf acct decision process
              to decide how much they trust your 'live' trading.

              The point is:
              Not all perf accts are generating real exchange orders.

              Your perf acct could be simulated remotely or those
              orders could be sent to the exchange -- it's up to
              Apex's risk management team how to handle your
              per acct orders.

              The difference is that Apex is a prop firm, and Apex
              guarantees to pay you all profits from your perf acct,
              even if Apex never sent (some or all) of your perf acct
              orders to the exchange. Remember, Apex doesn't have
              to reveal exactly how your profit was achieved, they just
              have to pay it.

              The presumption is that you, the trader, are not supposed
              to care about this sleight of hand risk management on
              whether your perf acct orders are simulated -- you're
              only supposed to care that you get paid. That's why
              Apex calls it a 'performance' acct, and not a 'funded'
              acct. You are paid your profits based upon your
              trading performance, regardless of their under-the-
              hood risk mgmt decisions, if any, that may have kept
              your perf acct in a simulated mode.

              My point is, if your measured times for an eval acct
              and perf acct are the same, I would presume your
              perf acct is being simulated. But if your perf acct
              times are different (probably worse), well, then, I
              would say those perf acct orders are probably being
              forwarded to the exchange, which could incur the
              extra overhead you've measured.

              You could always ask Apex about this if you find
              the measured time differences concerning.

              Just my 2˘.


              Comment


                #8
                this is a PA account, probably simulated cause it is new

                yes, I will monitor this as I go along...thanks

                being a scalper this is an important issue

                I've also traded a live Amp Futures account...with a worse latency (from Argentina, now I'm in Ecuador)...visually it is hard to tell 100 milliseconds difference

                but always good to measure the process

                will probably track this in an indicator and at least show it in an Output window

                Comment


                  #9
                  also, I noticed on an Apex Eval Account that more than once that Limit orders filled when the price was nowhere near the order....definitely a problem in their simulation...now I track all the submissions with time/price/Limit

                  Comment


                    #10
                    Hi bltdavid,

                    I've learned a lot about how Ninja works from your posts on this thread and others, thanks for that.

                    I currently trade a live Ninja account, as well as 20 Tradovate APEX PA accounts through Replikanto. In general they are all trading a 1 lot NQ.
                    I'm always torn between whether or not I should lead with my live account, or lead with one of the PA accounts.
                    If you were in this same situation, what would you do, or what factors would you consider in deciding how to choose your lead account?

                    Thanks

                    Comment


                      #11
                      Hmm, well, what experiments have you devised?
                      What are the results?

                      Me?
                      I'd probably trade the live Ninja acct as the leader.

                      Why?
                      Because this account, this entire account, is real
                      money -- and you'd want this account to be as
                      responsive (aka, trade with less slippage, etc) as
                      possible. The PA accts are important, of course,
                      but I'd probably favor the NTB acct as the leader,
                      since the money in that acct seems more dear.

                      More 'dear' because replacing your Ninja acct
                      would probably be more painful than replacing
                      the 20 PA accts -- but, then again, maybe the
                      20 PA accts are more dear. I dunno, those kind
                      of feelings are up to you.

                      -=o=-

                      Ok, well, let's try to think this through a bit.

                      First, try to measure the total time for a single
                      order in your Ninja account (no trade copying).

                      Then measure the time for an Apex PA account.
                      (again no trade copying) -- you can do this for
                      just one acct, or do this for all 20 accts, that
                      would be up to you.

                      Make these experimental trades at roughly
                      the same time, but don't overlap them, make
                      sure one exits before you enter and time the
                      next one.

                      Compare the times, are they roughly equal?

                      If there were any noticeable differences, I'd
                      be choosing the fastest acct as the leader
                      acct.

                      -=o=-

                      Here's another idea.

                      Since you have 20 PA accts (congrats!), do you
                      ever see any issues with just a few of the accts
                      showing slippage?

                      What I mean is, I presume Replikanto is coded
                      to perform replication in a fast simple loop. Thus,
                      the follower accts at the end of the loop might be
                      affected by the latency buildup of the accts ahead
                      of them.

                      But, I don't know for sure, I'm just kinda speculating
                      here. I've never seen their code.

                      Anyways, here is an interesting experiment:

                      Fire up two trade copiers, use the same leader
                      account for both (your NTB acct) but make each
                      copier replicate to just half of your PA accts.

                      The theory is -- you're shortening the internal
                      loop as the copier works through all follower
                      accts replicating the leader order -- I mean,
                      each loop is now only handling 10 followers.
                      This might help the accts at the end of the
                      loop avoid any latency/slippage issues from
                      the 'overhead' of being last in the loop.

                      The point is, two loops of 10 followers might
                      perform better than one loop of 20 followers.

                      You'd have to experiment and find out.

                      ​I'm presuming the overwatch on the same leader
                      can be established multiple times, using multiple
                      copiers. It is the follower accts that are critical
                      to doing this right -- you must take great care
                      to make sure each copier has a unique list
                      of follower accts.

                      You should ask Replikanto about the merits
                      of this approach.

                      -=o=-

                      Other ideas are, making sure your trading
                      machine has the fastest possible CPU, as
                      well as a huge amount of RAM, and the
                      absolute fastest possible SSD.

                      And then do you have the fastest possible
                      internet speeds for your location?

                      Those are just some extra ideas to think
                      about.

                      -=o=-

                      Actually, I'd probably have NinjaTrader running
                      on a VPS, esp a VPS co-located as near to the
                      exchange as possible.

                      Even if you trade manually, there are quite a
                      lot of benefits to be had from using a VPS.

                      As you approach this, be very methodical and
                      devise some experiments, and keep notes as
                      to your observations. Find ways to measure
                      the times, employ a variety of tools until you
                      get the measurement fidelity you want, ask
                      your VPS provider for some help in how to
                      measure round trip performance for an order
                      submission.

                      It is only through your historical observation and
                      measurement notes that you'll be able to discern
                      if any particular setup is more favorable than
                      another.

                      Of course, with a fast enough PC (whether a
                      local PC or a remote VPS) and fast enough
                      connection, it's very possible the leader acct
                      doesn't really matter -- you might just see the
                      same trade copier execution performance no
                      matter what you do.

                      Good luck!



                      Last edited by bltdavid; 03-09-2024, 12:05 PM.

                      Comment


                        #12
                        Thanks for taking the time to explain all that detail in a response, I really appreciate it.

                        What experiments have I devised?- Well up until recently I've never had to deal with this type of situation-- multiple funded accounts all getting copied to and consistently growing in value. I've been at this for over 2 decades, with winning years in '98, '99 and '08, but losing money every other year outside of those. So my recent success has only now started to make these types of questions relevant (in my mind at least).

                        And honestly I don't even know how to run these latency tests on my Ninja platform or PC. So I'll be reading through these threads and others as time permits, to try and figure all of this out.

                        For me, the main thought I had on the subject of using the Live or Apex account as the leader was this: If I trade the live account first, this might cause more likelihood of slippage on the Apex followers since my live fill (on a market order long for example) would take out an actual resting order on the Ask, before the Apex accounts get triggered. Leading with Apex (if on Sim and not being copied by Apex) would not actually take out any real resting orders before my live account triggers. But this whole idea is probably more of a concern if trading something like a 10+ lot on NQ, and not so much with just my current 1 lot.

                        Regarding which is more 'dear' to me, that's a great question. In many ways, the 20 Apex PA accounts (being all past the withdrawal threshold) are essentially real money at this point, and represent 20x more $ than the 1 lot I trade live. So they might be more important overall, but I do agree that actual live personal accounts tend to hit differently than a PA account if blown.

                        On the 2 trade copier idea, I actually trade the 20 PA + 1 Ninja Live on my main PC, but also use another computer where I have a 2nd Replikanto copying on Network Mode. On that instance of Ninja I have 6 Bulenox Rithmic Accounts (3 funded and 3 Eval currently). (Yes, I'm trying to make up for lost time I guess by maxing out my accounts where I can)
                        I do notice that there are quite often 6-7 ticks or so differentiating the fills across all of these 27 accounts. But sometimes a lag can be in my favor, since I'm trading fairly discretionarily- using market orders to get in and out for most trades- as opposed to getting out at a fixed stop or target. So often a lag comes in my favor, especially on the exit of a winning trade (which is why I'm not overly concerned I think on this issue, but was curious if someone with your knowledge of the tech side would have an opinion on it)

                        Regarding my PC, it's a AMD Ryzen 7 1700X Eight-Core Processor 3.71 GHz with 64G of RAM. Again, I'm not really a tech guy, just copied and pasted that here. Would you consider this sufficient or minimal for what I'm doing?

                        The posts earlier in this thread about calculating latency on Apex PA vs Apex Eval accounts to hypothesize if an account is being copied to the live market was really interesting and something I'd never thought about. Also I'm intrigued by the idea of a VPS but need to understand it more.

                        Thanks again for your time. A lot for me to learn on these subjects I know!​

                        Comment

                        Latest Posts

                        Collapse

                        Topics Statistics Last Post
                        Started by Segwin, 05-07-2018, 02:15 PM
                        14 responses
                        1,788 views
                        0 likes
                        Last Post aligator  
                        Started by Jimmyk, 01-26-2018, 05:19 AM
                        6 responses
                        837 views
                        0 likes
                        Last Post emuns
                        by emuns
                         
                        Started by jxs_xrj, 01-12-2020, 09:49 AM
                        6 responses
                        3,293 views
                        1 like
                        Last Post jgualdronc  
                        Started by Touch-Ups, Today, 10:36 AM
                        0 responses
                        12 views
                        0 likes
                        Last Post Touch-Ups  
                        Started by geddyisodin, 04-25-2024, 05:20 AM
                        11 responses
                        62 views
                        0 likes
                        Last Post halgo_boulder  
                        Working...
                        X