IB Symbol Limit problem

[kdoren]

I'm testing a strategy which I want to watch many pairs of stocks, but which will generate relatively few trades. For example, it might watch 100 pairs of stocks (200 instruments) and generate maybe 5 trades per day. I'm using IQFeed as my data feed (connected first), and Interactive Brokers for execution.

I enable the strategy in the "Strategies" tab, select an instrument list and an IB account, and NT creates a strategy instance for each instrument on the list. So far so good.

When I select the strategies and enable them, after enabling some of the strategies, NT7 starts generating a pair of error messages in the log for each subsequent strategy instance it tries to enable. The error message is: "Max number of tickers has been reached 101"

I don't understand why I'm getting this message, it's clearly an IB limit. I should be getting the data feed from IQFeed which has a 500-symbol limit. I don't want to be getting any data from IB at all, yet I'm hitting the IB limit of 100 symbols.

How can I make sure I'm getting the data from IQFeed so I don't hit the IB limit?

Thanks,

Kevin

[NinjaTrader_Jason]

Hello Kevin,

If you connect to IQFeed first and IB second, all data should be loaded from IQFeed as per the link below.

http://www.ninjatrader.com/support/helpGuides/nt7/multiple_connections.htm

Can you please post your latest log and trace files that reflect the loading of this data. You can find log and trace files at the following locations.

Start Menu--> My Documents--> NinjaTrader 7--> Log--> log.20100723 for today. (log.yyyymmdd for any other date)

Start Menu--> My Documents--> NinjaTrader 7--> Trace--> trace.20100723 for today. (trace.yyyymmdd for any other date)

In addition, if you solely connect to IQFeed and load all stocks in a Market Analyzer window, do all stocks receive data?

[kdoren]

I am currently using IQFeed 4.7.1.3 and NT7b19, but I have seen this problem on earlier versions of both. IQFeed stats shows Maximum Symbols of 500, currently using 150.

Is it possible that this error messages is coming from NT7 because it sees >100 symbols while an account connection is open to IB? If so that is a bug if IB is not supplying the data.

The Trace file shows that NT7 is requesting the data from IQFeed. All requested symbols load fine, until they start coming paired with error messages like this:

2010-07-23 03:16:28:730 (IQFeed) Data.Bars.GetBarsNow: instrument='ITT' from='2010-07-22' to='2010-07-23' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0 session='US Equities RTH'
2010-07-23 03:16:31:374 ERROR: Max number of tickers has been reached 101

Similar thing in the log file:

7/23/2010 3:16:30 AM|1|128|Enabling NinjaScript strategy 'PairTradeDaily4/8ccc9636644e4a7691b1bb0d31a6478f' : On starting a real-time strategy - StrategySync=WaitUntilFlat SyncAccountPosition=False EntryHandling=AllEntries EntriesPerDirection=3 StopTargetHandling=PerEntryExecution ErrorHandling=StopStrategyCancelOrdersClosePositions ExitOnClose=False Set order quantity by=Strategy ConnectionLossHandling=KeepRunning DisconnectDelaySeconds=10 CancelEntryOrdersOnDisable=False CancelExitOrdersOnDisable=True MaxRestarts=4 in 5 minutes
7/23/2010 3:16:31 AM|3|4|Max number of tickers has been reached 101
7/23/2010 3:16:31 AM|3|4|Max number of tickers has been reached 101

[NinjaTrader_JoshP]

kdoren,

Are you sure you are connecting to IQFeed first? I tried this on my end and do not receive the max instrument message.

1. Start NT, blank workspace
2. Connect IQFeed
3. Connect IB
4. Open Market Analyzer
5. Add S&P500 list --> all instruments load and continue to stream data --> no errors

[kdoren]

I connected to IQFeed first.

There is no problem when using the market analyzer. It seems to be specific to strategies. In the "Strategies" tab, the strategies have their "Connection" listed as "Interactive Brokers", presumably because the Account is an IB account. I believe the data is really coming from IQFeed (IQFeed says I am using 150 symbols, and the trace file indicates IQFeed also).

So I can't imagine where the symbol limit error is coming from, unless NT7 is incorrectly generating it because I have >100 symbols active that are connected to IB accounts, even though I'm not using IB for the data.

[NinjaTrader_JoshP]

Please clarify how you have ">100 symbols active connected to IB"? What constitutes "connected to IB"? Thanks.

[kdoren]

I have strategies enabled which collectively use >100 symbols.

All of those strategies have an "Account" parameter (i.e. where the trades get placed) which is set to an Interactive Brokers account.

After the strategies are enabled, the "Connection" column on the strategies tab shows "Interactive Brokers".

Once enough strategies which use an IB account have become enabled, I get the "Max number of tickers has reached 101" message for all subsequent strategies as they become enabled. FYI symbols used on charts or in the market analyzer have no effect, it has to be strategies which use an IB account.

I connected to IQFeed first, and the trace files show the data requests going to IQFeed.

The strategies show "green" on the strategies tab, even the ones which triggered the error message. I don't know if any of the strategies which generated the error attempted to trade. It's not clear to me if the error message is merely an annoyance (i.e. maybe the strategies are all getting data from IQFeed just fine, and any trades via IB would work as long as the number of traded symbols is <=100), or something of greater concern (i.e. trades won't work).

[NinjaTrader_JoshP]

kdoren,

Reason this happens is because strategies enabled on strategies grid pulls market data from the connection where the account is on. Having 100 strategies means you are already pulling 100 instruments. Adding more means you surpass the IB limit. This data pull is required to actually run your strategy.

[kdoren]

Huh? You have clearly stated many times that the data feed for an instrument comes from the first connection opened which supports that kind of instrument.

This is clearly a bug that needs to be fixed. Otherwise what is the point of supporting multiple connections?

Are you saying that if the strategy is attached to a chart, the feed will come from one place, but if the strategy is in the strategies grid the feed will come from a different place? That makes no sense at all.

I need my strategies to run in the grid (I can't be opening 100 charts) and get their data from IQFeed (which is a better feed with a 500-symbol limit) and issue trades through IB. It was my clear understanding based on your published statements that this was possible. Please fix it.

Your support of multiple connections really needs to be thought through better. My suggestion would be that in the configuration for each connection, for each instrument type it should be possible to enter a 1-digit "Priority" for both real-time and historical (with 0 meaning "never") In my case IQFeed would be priority 1 for stocks for both real-time and historical. IB would be priority 2 for real-time and 0 for historical. The connection order wouldn't matter, and you wouldn't need the "back-up" configuration setting. If 2 connections had the same priority, you could use the connection order.

[NinjaTrader_JoshP]

Yes, data comes from the primary data provider, but when you actually run a strategy it can only do so by opening the connection to that instrument from the brokerage account as well. As such having 100 instruments = you opened 100 instrument connections on the brokerage feed and as such you hit the limit. You simply cannot run more than 100 symbols with IB.

[kdoren]

That's very unfortunate. That means there's no way to watch > 100 stocks on IQFeed (500-symbol limit) and trade only a few ot them on IB, because NT7 loads every symbol being watched into IB even though most will never be traded.

Does it really have to be this way? Is there any way to defer loading the symbol into IB until a decision is made to enter an order? Any hints would be greatly appreciated. And please consider changing this behavior.

Thanks,
Kevin

[NinjaTrader_JoshP]

Kevin,

Unfortunately there is no way you can run a strategy tied to a specific account without establishing an IB connection to that instrument. Running a strategy requires a price feed and an order feed. It is unfortunately not feasible to say let us only connect to IB's order feed for this strategy when the strategy actually wants to trade and then disconnect after it places its trade. By the time a connection can be established in such an ad hoc scenario, you would be many ticks away from your entry signal resulting in considerable slippage.

[kdoren]

So, to clariify, there is no way for a strategy to issue a trade for symbol XXX with IB, unless that strategy opens a connection to IB for instrument XXX at Initialize() time.

If, for example, I wanted to make a strategy which just sat there waiting for a message to execute a trade (for an unknown symbol which is contained in the message), there is no way to do that? Completely impossible? (It might be slow, but I would think there is some way for a strategy to launch another strategy which would then initialize itself for the symbol which is now known?)

Thanks,
Kevin

[NinjaTrader_JoshP]

Kevin,

There is unfortunately no way in NinjaTrader to do this. You cannot open connections in an on-demand sense.

[aviat72]

kdoren:

IIRC you can pay IB a nominal fee to get more symbols. And if you generate enough commissions or have a large account the number of symbols also goes up.

I agree with Josh here, that it is not unreasonable for NT to request a data-feed from the broker where the trades are happening.

I have also requested the other enhancements you have asked for. Ideally I should be able to specify the preference on a per-symbol basis, with global groups. I subscribe to IQFeed for futures, and for forex would like to use the IB feed. However I can not do that right now.