Connectivity problems

[adamus]

Both affected PCs synch regularly with time.nist.gov as per the normal regular Windows config.

I am not quite sure what you hope to ascertain through a test with a different ISP. You have already told me that nobody else has reported this error, so what good will it bring if I can run my strategies under a different set-up? It won't change the fact that my current set-up is semi-disabled by the issue.

This cannot be so difficult. As the InteractiveBrokers support person said -

In this particular case, base on the message on the log file, your API program must validate the time, duration, and barsize combinations before requesting historical data.

The core issue is within your API program, not the TWS. I strongly recommend that you consult with the third party vendor to resolve the issue.

So NT is making requests from IB for data using parameters that don't make sense, or similar.

Thanks in advance for getting your programmers with experience in TWS API coding to look into it. I don't blame you guys for getting it wrong, I know the TWS API is a monster, but you've got to look into this seriously before torturing me more with the stuff about the internet connection.

[NinjaTrader_JoshP]

If NT sent incorrectly formatted requests every single IB user would be influenced. This is not the case and it is unfortunate that only you are reporting running into this. At this point in time the symptoms that have been isolated strongly suggest simply connectivity issues with either your home network or your ISP to IB servers. Timeouts/extreme latency between data requests due to connectivity issues could definitely cause a seemingly improper request as IB thinks it is. All we can suggest is for you to try on a different network setup. This means trying to swap out all variables; try different equipment, different ISP, etc. Would probably be easiest to just trying to go over to a friend's or relative's house and trying on their network setup.

[adamus]

Timeouts/extreme latency between data requests due to connectivity issues could definitely cause a seemingly improper request as IB thinks it is.

Basically the way that IB will read that statement is this:

Timeouts/extreme latency between data requests due to connectivity issues represent an added layer of complexity that NT would have to deal with.

Hopefully you see what I mean - NT simply isn't handling it.

Secondly personally I do not see how connection problems should influence what you are requesting - other than to delay or stop your requests.

Could you please let me know what your TWS API programmers say on this issue?

I will work on checking my internet connection and trying to isolate any potential problems.

Lastly, you still didn't say how using somebody else's setup to run NT & IB will benefit me. Again, you say the whole of the rest of your customer base doesn't have this issue so what reason is there to expect that using someone else's setup would shine a light on what the issue is?

[NinjaTrader_JoshP]

adamus,

Not sure what you mean "isn't handling it". There is nothing to "handle" if the issue is simply connectivity issues that needs to be addressed from a hardware perspective. There is nothing wrong with the request format NT sends out. If the request format was incorrect thousands of customers would have the exact same issue, but the unfortunate reality is that only this particular isolated case here exhibits this behavior. From the sound of the issue you are having you are either having some packet loss or other such type of issue with your network. There is nothing NT can do to try and fake in packets for you or something to that effect because they keep getting dropped. If NT sends out a request and it just disappears/corrupts into the internet unfortunately there is nothing NT can do about that.

Of course all of this is speculation as we would not know or be able to definitively diagnose your network for you, but all your stated symptoms point to telltale signs of network/connectivity issues. This is why we have asked you to try on a different network multiple times. Trying on a different network means you hopefully remove the network/connectivity issues causing you all these problems from the mix. When you confirm it works on a different network and ISP then you can definitely know that the issue is with your particular setup. You would then unfortunately need to replace your network. Whether that is your modem, your router, maybe some ethernet cables, maybe your ISP, we could not tell you that as we are not network experts to be able to help pinpoint the exact point of failure.

[adamus]

Just another thing that occurs to me: after 23:00GMT+1 the problem is far far more manageable than before 23:00. During the morning it is generally tolerable as well. Perhaps there is something to do with the forex session in NT that slowly gets more and more out of sync during the day, to the point that it fails completely during the US afternoon session, until 23:00 when it resets again.

Also I have contacted IB with a copy of your message in an attempt to get them to be more helpful with the issue.

Thanks

[adamus]

I just read your last message after posting #50.

Please would you confirm that you do not intend to put this issue to your TWS API programmers? At least then I will know where I stand.

[NinjaTrader_JoshP]

adamus,

Please note that development has already taken a look at this thread and we are in agreement that the internet/connectivity hypothesis needs to be isolated and locked down first before we can continue any further investigation.

Please first try on a different network and then let us know the results.

[adamus]

Re running on a different network - this has been done already at least twice by your own staff when I have been trying to resolve other issues. Do you now expect me to change my hardware and my ISP?

It seems you are merely repeating your request until I say "it worked somewhere else" and then you can wash your hands of the issue - problem solved for you guys, no need to get your programmers to resolve that prickly issue with the TWS API and the IB error.

Like I said, I've been there, I know what a problem it can be. If you get this program out of beta with this bug still in it, you run the risk that many of your customers will come back to you with the same issue.

The least you should do is fix the IB connection logic to throw correct errors when IB is having connection problems, if that is indeed the problem, which is not a definite fact. Just handling the issue the current way isn't robust and has caused me a headache since day one.

You say you are not experts in networking and that it's not in your remit to help me with that - fine I don't have a problem with that. What I do have a problem with is that you are effectively stonewalling and in denial about the incorrectly formatted requests you are making across the TWS API.

If I replace my ISP and change my hardware and this solves the problem for me, then great! I would be back in business. Since I would never then find out what the actual problem was from IB, because you never fixed the bad requests from NT, then it might just happen again, to me or any other of your customers - the real solution might be any one of a number of simple internet configuration changes, but we would never know and you would be telling people to change ISP and get new hardware.

Isn't this what beta testing is meant to be sorting out?

[adamus]

From the sound of the issue you are having you are either having some packet loss or other such type of issue with your network.

Currently faultless internet connection:
Ping statistics for 208.245.107.9:
Packets: Sent = 100, Received = 100, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 95ms, Maximum = 107ms, Average = 96ms

Still getting the error from IB about the "Invalid time period":

JU 22:51:17:227 JTS-CCPDispatcher-10586: CONFIG | WARNING | Invalid time period, using default | -1 | 0 | null

[NinjaTrader_JoshP]

adamus,

We realize this is frustrating for you and it is frustrating for us as well that it is not working the same way for all of our customers. The error message you are running into is not the first time we have seen this "HMDS data farm connection is broken" message. A historical case of this was resolved when the user replaced their router. We have also once again reviewed the code and would not see where an improperly formatted request could cause any such data farm connection issues. Based on the fact that no one else is running into this, code review, and the resolution of the historical case, at this point in time all things point to some intermediary factor and all we can suggest is for you to try on a different network.

Please note that IB log message is irrelevant as it would not cause the data farm connection to break.

[adamus]

Hi Josh,

I think we've established that my strategies will happily download historical data when run on other machines. Your support colleagues have run the strategies. You requested multiple times that I try this, so now you know it's possible, what does it tell you?

What more can you tell me about this error that appears in the IB logs? You have done a code review with your programmers. What do they say? Is it irrelevant? Is the information I provided to you from IB support about this also irrelevant?

Initial Description: HMDS data farm connection continually breaking

Response from rayros at 04-Oct-2010

Hello There,

Unfortunately, we cannot ignore the warning message. This is hard-coded
on our server side. It is a standard protocol if the request is
invalid, then the designated server will throw some error messages.

Regards,

Raymund

IB API Support

We have also established that this error occurs even when the IB connection is stable, but this shouldn't give you an excuse to ignore it. What we haven't established is whether all of your customers may well see this error as well if they turned on the logging at the appropriate level in IB. That seems quite likely to me.

Another point of agreement is that you, as programmers and not network experts, are unable to give further advice on network connectivity. This would be fine except for the fact that IB will not address the network problem while NT is making bad requests. Also I think I am correct in spelling out your position towards IB: you think they are wrong to highlight the "Invalid time period" error and they should give me advice how to improve the connection to their HMDS.

My opinion is that you should be taking this error far more seriously. I hate to tell people their job but in this case it is unavoidable, unless you will confirm to me that IB Support are wrong in their analysis.

One of your customers with a similar problem was able to resolve the situation by replacing the router. Does this mean now that all customers with suspected network problems will be advised to replace their router? Wouldn't it be better for your reputation and your customers' sanity to discover what the actual problem is rather than rely on this unprofessional guesswork?

In case you think I am not doing anything at my end, I'd like you to know that I am spending a large percentage of my working day trying to resolve this. I would like to think that the two software companies whose products I pay for are actually trying to help me rather than trying to put the blame on the other.

So far what I have done to eliminate the IB connection issue (which would still leave the issue with the "Invalid time period"), I have:

- replaced the DSL modem
- requested broadband line checks by the ISP
- request help on disabling non-essential network protocols (e.g. TCP-IPv6)
- set up network monitoring to check ping results, HTTP requests, DNS request on multiple servers at different points between myself and the IB server
- ran various different diagnostics on an intensive basis e.g. tracert, ping, firewall logging, etc

Despite all of this I still see it as unquestionable that your responsibility lies in fixing that IB error, or giving me a reason why not that IB will accept. You try telling them that the error is irrelevant! I would be happy to organise it if you would give me the times when you are available for a conference call, although of course there is no reason to suspect that your attitude would be any different over the telephone than it has been on this forum.

If the strategies I ran weren't profitable or if I was just starting out or if I didn't see the potential in NT7 beta if only the outstanding issues can be resolved or if I didn't have a work-around for obtaining historical data, then obviously I would drop NT immediately. This will not be the case for the majority of users who come to you in the future if you release NT7 with that issue in it still.

It would be great to hear something positive from you.

[poeds]

It seems to me that NT is sending valid requests to IB's server, as has been demonstrated from other locations using the same NT build. Those requests are then becoming corrupted in transit from your client to the the IB server. The IB server is then rejecting the requests as invalid. Although the internet connection may appear stable and pass various diagnostics it would appear that there is instability at times that particular packets are sensitive to.

[adamus]

Let me correct you: the instability of the connection with the IB server is time-of-day-dependent, however the appearance of the "Invalid time period" error in the logs is not. It occurs with every request.

In fact I fired up NT6 to see if it was happening there and it was.

Also, it's kind of counter-productive here at this point where I'm trying to persuade the NT support guys to do something, if you make suggestions such as particular packets being sensitive to instability. I'm trying to get some solid answers and the guesswork isn't helping.

I can't convince my ISP nor InteractiveBrokers to look into this by citing something so vague.

I need some focus like ICMP and packet corruption checking testing. Not that I know how to test that, but I probably will soon, and the chances are, if I do find and solve a network bug, then this NT bug will disappear into the woodwork to come back and bite me again later.

Sorry to rant at you but this whole thing is getting way off track.

[poeds]

dont worry ab rant

u mentioned ipv6 in post 56
i wonder did u disable it via that link I put up b4?

[adamus]

No, I followed the menu selections to disable it manually.

I just did that now but I have to reboot for it to take effect and I'm currently got NinjaTrader wrapped up in cotton wool because I've got 10 positions on via the strategies I'm running.

However it's all a bit more complicated than that.

First up, what I just said applies to my live workstation running Win7-64, but my dev box with XP never had IPv6 setup and the problem is the same.

Not only that but the actual DSL modem and gateway machine are linux so I'm looking into how best to disable IPv6 wholesale for the whole LAN, via the DHCP server.