NinjaTrader + Rithmic = Not updating positions

[Rudmax]

Hello Rudmax,

Our team has contacted Rithmic who has added additional logging to their brokerage end.

Once a user who has experienced the behavior writes into support[at]ninjatrader.com, we will again contact Rithmic to review the logging for the transactions.

However, if a user is experiencing high latency or connection issues and updates are not being received, there would not be much that can be done about that. We recommend using a VPS located near the brokerage servers to avoid interruptions in connection.

Since so far there has been no solution from NinjaTrader for the failure of Rithmic APEX connections to update their positions in NinjaTrader, I would like to know if there is any NinjaScript code that forces broker positions to be updated in the NinjaTrader tool, and developers could use this code to call it immediately after filling TP or SL exit orders. Please suggest something.

[NinjaTrader_ChelseaB]

Hello Rudmax,

Our developers have been looking into the issue but a correction, if made, would only be in a new release of NinjaTrader. So you would need to wait until the next release.

That said, my thoughts are with the trade copier code. I suspect the trade copier is doing two things that can cause an issue.

1. Possibly the developer is not making copies of collections with .ToList() and doing loops over the copy collection.
Not doing this can cause too much work on the account/order thread and cause updates to be missed.

2. The developer is doing work in an event handler method without passing the work to a new thread. This also could be causing too much work on the account/order thread and causing updates to be missed. While multi-threading in general is not officially supported by NinjaTrader, in this case it would be a practical solution to prevent the account/order thread from missing updates due to being overworked.
Possibly even a few dozen milliseconds between order submissions might correct the behavior as well.

While I'm not on the development team and don't have their insights, I suspect that the team is trying hard to come up with a way to handle this issue with inefficiently written code in an addon, which may take time.

That said, in my opinion, and if my hunch is correct, the issue could quickly be solved by author of the script.

I could be completely incorrect. But by observing my own code and observing what occurs with manual and atm orders, it's very possible.


"I would like to know if there is any NinjaScript code that forces broker positions to be updated in the NinjaTrader tool, and developers could use this code to call it immediately after filling TP or SL exit orders. Please suggest something."

There is no code that I am aware of to force a brokerage to send an information update, and there is no code that will force NinjaTrader to process an information update if the CPU logical processor is overworked.
Unfortunately I cannot suggest something on that end. I can only suggest better written code in the addon.

[Rudmax]

Our developers have been looking into the issue but a correction, if made, would only be in a new release of NinjaTrader. So you would need to wait until the next release.

Your message was a ray of hope for us!

That said, my thoughts are with the trade copier code. I suspect the trade copier is doing two things that can cause an issue.

What strikes me is that a third party program can cause this problem in NinjaTrader, causing it not to update positions correctly. What fascinates me most is that this only happens with Rithmic APEX accounts, while it does not happen with other account types.

1. Possibly the developer is not making copies of collections with .ToList() and doing loops over the copy collection.
Not doing this can cause too much work on the account/order thread and cause updates to be missed.

In our case, we always use .ToList() for loops in the collections, because without it the program is susceptible to errors such as: InvalidOperationException: Collection was modified; enumeration operation may not execute.

2. The developer is doing work in an event handler method without passing the work to a new thread. This also could be causing too much work on the account/order thread and causing updates to be missed. While multi-threading in general is not officially supported by NinjaTrader, in this case it would be a practical solution to prevent the account/order thread from missing updates due to being overworked.
Possibly even a few dozen milliseconds between order submissions might correct the behavior as well.

As this is a third-party add-on, we believe that it has no influence on the updating of account positions in NinjaTrader. In any case, it could in no way affect the updating of positions performed by NinjaTrader itself. If the way to use the event handler for accounts/orders with a lot of work in the same thread, in that case, it should only affect the add-on and not the NinjaTrader itself. I don't think this could be the problem, not least because accounts of other types, such as Tradovate, don't have this issue (not showing the most current position status). However, we are available to meet with the developers of NinjaTrader to provide snippets of our code for evaluation.

While I'm not on the development team and don't have their insights, I suspect that the team is trying hard to come up with a way to handle this issue with inefficiently written code in an addon, which may take time.

That said, in my opinion, and if my hunch is correct, the issue could quickly be solved by author of the script.

It would be a dream for us to solve this ourselves, not least because this kind of problem will have a anniversary. If we can solve it via code, that would be great, although we're not sure it's possible due to the factors mentioned above. We are available to meet with the NinjaTrader developers.

I could be completely incorrect. But by observing my own code and observing what occurs with manual and atm orders, it's very possible.

We may be mistaken, but we believe that a solution to the problem would be beneficial for all of us. It would help to reduce the number of open tickets and dissatisfied customers on both sides. We are committed to resolving this situation, as we know you are too.

[backtester831]

What fascinates me most is that this only happens with Rithmic APEX accounts, while it does not happen with other account types.

Maybe Apex Rithmic server is overloaded on their end?

[Rudmax]

Our developers have been looking into the issue but a correction, if made, would only be in a new release of NinjaTrader. So you would need to wait until the next release.

With the next NinjaTrader release, which will update the Rithmic API to version 13.4.0.0, can we say that this might solve the problem?

[NinjaTrader_ChelseaB]

Hello Rudmax,

Our developers are looking into the issue.

Unfortunately, I will not be able to say if a correction will be ready by the next release.

Once the release is out, we'll check the release notes, and if listed we'll have users test for the behavior.

[Finley]

Maybe Apex Rithmic server is overloaded on their end?

I don't think it's an APEX Rithmic issue because some traders in our group use Sierra Charts or Quantower and don't have problems with positions still showing as open.

Another clue is that I have three different copy trader copiers—Apex, Replikanto, and ADTS Smart Trade Copier—and the same issue appears regardless of the copier. So, it would seem that all roads point back to a Rithmic-Ninjatrader interface issue.

[Rudmax]

I don't think it's an APEX Rithmic issue because some traders in our group use Sierra Charts or Quantower and don't have problems with positions still showing as open.

Another clue is that I have three different copy trader copiers—Apex, Replikanto, and ADTS Smart Trade Copier—and the same issue appears regardless of the copier. So, it would seem that all roads point back to a Rithmic-Ninjatrader interface issue.

Thanks for your feedback!

Yes, we understand it's a problem between NinjaTrader and Rithmic-type accounts, but not necessarily a problem with Rithmic but with NinjaTrader itself when it comes to updating the positions of several accounts at the same time.

Unfortunately, we are still waiting for this issue to be fixed by the NinjaTrader.

[defa0009]

still waiting for this issue to be fixed by the NinjaTrader.

I don't see that issue in NinjaTrader when using the Lumens Trade Copier for NinjaTrader.

[dragutin]

I do not recommend every one to trade with trade copiers.
i had this week same problems like in this thread explained from many users. Ninja trader is not programmed for multiple accounts execution ect …..
Same with rithmics i was up 500 doller went short was not able to close position with Replicanto than i opend rithmic pro and closed the position and i was down 200 total Although market never went against me horrible.

[ntcoder256]

Hello Rudmax,

Our developers are looking into the issue.

Unfortunately, I will not be able to say if a correction will be ready by the next release.

Once the release is out, we'll check the release notes, and if listed we'll have users test for the behavior.

Hi, was this ever addressed in any updates? I'm am on the latest version and still see it. I'm using Apex, with both Replikanto and Apex trade copiers. Happens very consistently, I've lost several accounts from this.

It seems like there's a bit of a circular firing squad situation going on with the customer stuck in the middle.

I don't like Rithmic, I sometimes have problems closing positions left open even via RTrader and there's no resolution for that either.

On the other hand Ninja Trader is a great company, I would hope maybe Ninja Trader could be the bigger party and "fix" it even if it's really something Rithmic should do...because there's no outreach to them and I think they could care less.

please consider it.

thanks

[NinjaTrader_ChelseaB]

Hello ntcoder256,

Our developers are looking into the issue and the case is still open with them.

As there are 3 parties involved, it can be challenging to precisely identify the specific cause.

I have my own suspicions that the issue comes from addon code. I've requested that our business development reach out to Apex to assist me in investigating but unfortunately I have not received any information back.

See post # 78 where I've discussed my suspicions.

The gist is that the Account thread is the thread in NinjaTrader that creates and sends orders, and processes order, execution, and instrument updates. If the account thread is overloaded, this causes missed updates and missed order submissions because it can no longer actively listen for updates until the processor is freed up.

This is similar to an issue calling a thread sleep on an instrument thread which causes the entire platform to stop receiving data. After the thread is freed that data does not get processed because the updates were missed.

Another suspicion I have is the addon may be exceeding the order action rate for the connection type / brokerage.


An analogy would be trying to move a swimming pool amount of water through a straw. The straw representing a single logical processor that is processing updates, creating orders, and transmitting these, the water representing the amount of instructions including logic in the account event handlers as well as the background order processing and brokerage communication work. Trying to move 100,000 liters of water through a straw in less than 1 millisecond will create so much pressure the straw fails (the straw would burst, but in the analogy the processor just locks up because the L2 cache has no more free slots).
However, if you were to pace the transmission and send 1 liter of water through the straw every 5 seconds, the straw would have a much better chance of keeping up. The goal would be to come up with a procedure that works with different qualities of straw strength and different amounts of water that works well in most situations.


If the issue is with the addon, it might not be possible for our development team correct on our end.
However, without any feedback I don't have a way to confirm these suspicions.

I've created a test script which duplicates orders to multiple accounts, which I've provided to our business development team, which is what I have requested feedback on.

In my personal testing, I've stress tested the test script extremely hard with no issues on a single Rithmic simulation account and with multiple NinjaTrader demo accounts, however I do not have access to a large number of Rithmic accounts to test this myself.

I'll happy share this here if you would like to do some testing on my behalf. The insight gained could give myself and our developers more information to tackle the issue.
PaceTradeCopyAddonExample_NT8.zip


I would like this tested on as many accounts as possible. My hope is for someone with 100+ accounts could test to see if there are limits to the number of accounts and limits to the pace of the submissions.

Requested Testing:

• Add several (as many as possible) live demo (or live funded) accounts to the recipientAccounts collection on lines 127 to 131.
  • Test if there is a specific amount of accounts that can introduce issues
    • Test 1 additional account, 2 , 3 , 5 , 7, 10, 20, 50, and 100 additional accounts if possible
    • Note how long it takes for the last account to fill an order and how far from the fill price of the first account this is
• Test different delay milliseconds between an order submission
  • The timerInterval integer on line 112 holds the number of milliseconds for the timer Elapsed event
    • Test adjusting this to 1ms, 2ms, 3ms, 4ms, 5ms, 6ms, 7ms, 10ms, 15ms, 20ms, 50ms.
• Test multiple instances of the window being open and if they how they conflict
  • Test multiple instances on different accounts but the same instrument
  • Test multiple instances on the same account but different instrument
  • Test multiple instances on the same account and same instrument.
    • It would be expected the position is doubled, but we specifically want to know if the updates are being received correctly

That said, even if I don't get the feedback I need, our development is still actively investigating.

[ntcoder256]

Thank you SO much for sharing this and working on this. I will absolutely go through your test cases as best I can and provide feedback, I am testing it already now.

It may be a little hard to test some of the live cases because this does not copy ATMs (I'm not expecting it to I get you're just trouble shooting and this is minimal test code) and I rely on a runner and trailing stop every trade. However I will still try some live testing with static profit targets, and frankly if this test code is more reliable to get me out of positions than the commercial ones I'm using I'll just switch to static profit targets.

One last note you mentioned you hadn't heard from Apex, if you haven't already you might reach out to FlowBots who makes Replikanto

https://flowbots.ninja/contact

They're actually the ones who referred me to this thread, after I contacted them regarding this issue. I've found their support very fast and responsive and they really seem to care about their products. Also I'm not sure if it's true but I've heard the Apex copier is made by FlowBots.

Best regards

[NinjaTrader_ChelseaB]

Hello ntcoder256,

"It may be a little hard to test some of the live cases because this does not copy ATMs"

The script was written to duplicate ALL order submissions, including those from ATMs. It does not replicate order modifications or cancellations because it's only meant to be a concept test and not full trade copier addon for use, but stop and target order submissions from ATMs should be duplicated.

"and frankly if this test code is more reliable to get me out of positions than the commercial ones I'm using I'll just switch to static profit targets."

If the test succeeds then we know a little more about the issue and any one of us (the platform, connection technology, or brokerage) may be able to make improvements with the knowledge gained so the entire industry benefits because the issue is mitigated for all of our mutual users.

"One last note you mentioned you hadn't heard from Apex, if you haven't already you might reach out to FlowBots who makes Replikanto"

I will relay that to our business development team who is the bureaucratic interface between my team and the wider ecosystem team of vendors.

[NinjaTrader_ChelseaB]

Hello ntcoder256,

May I confirm you are using NinjaTrader 8.1.4.2?