Loading data issues

[bayofislands]

Hi,

I am having repeated issues trying to get historical tick data over modest time frames. (5 day increments)

All attempts to "Abort" cause the system to hang.

If I wait for 40 minutes or so during the "loading data" I occasionally get an Error on loading chart data for 'ES 12-09 Globex' Unable to retrieve data

From the trace:

2009-12-03 11:44:25:265 (Mirus Futures) Data.Bars.GetBarsNow: instrument='ES 12-09' from='2009-11-12' to='2009-12-02 23:59:59' period=1 Tick splitAdjusted=False dividendAdjusted=False bars=0
2009-12-03 11:48:30:015 (Mirus Futures) Hosted.Adapter.OnAccountItemNow: account='donotuse' fcm='Dorman' ib='Sim' itemType=CashValue value=*****
2009-12-03 12:04:57:828 Connection to NinjaTrader data server lost: An existing connection was forcibly closed by the remote host
2009-12-03 12:04:57:828 ERROR: Connection to NinjaTrader historical market data server lost: An existing connection was forcibly closed by the remote host
2009-12-03 12:05:07:828 Server.Client.Loop0-1: 64.202.118.144/31654
2009-12-03 12:05:08:265 Server.Client.Loop1-1

As with comments in previous posts re: "loading data", a progress meter showing that data is actually being loaded and from what source would be helpful.

Cheers,
Paul.

[NinjaTrader_JoshP]

Paul,

Are you able to receive streaming data from the Control Center for ES 12-09 when connected to Zen-Fire? Have you made any changes to the instrument in the Instrument Manager?

[bayofislands]

Hi Josh,

Yes I'm able to receive streaming data. I haven't changed any instrument settings. It fails when downloading historical data for other instruments too, ie 6A 12-09.

My work around was to reduce the look back period to 3 days and then keep extending it by 3 days in the "data series" till I get to the period I'm after. A very tortuous approach. My real concern though is that it should hang the application on "abort", whatever else is going on.

Thanks for your help,
Paul.

[NinjaTrader_JoshP]

Paul,

I am not sure what may or may not be going on at this point in time. Please try a completely fresh reinstall of your NinjaTrader and ensure you are on 7.0.0.5. See if that helps. Thank you.

[bayofislands]

Hi Josh,

Yes I'm running 7.0.0.5

I'll do a re-installation shortly and let you know of the outcome.

Cheers,
Paul.

[bayofislands]

Hi Josh,

Renamed local directories NinjaTrader 7, NinjaTrader 6.5, NinjaTrader Backup

Uninstalled NT 6.5 & 7.0

Restarted OS

Installed NT 7.0.0.5

Launched NT 7.0.0.5

Put in NT Licence Key and Mirus connection details

Used instrument manager to add 6E & 6A to default instruments

Selected File - New - Chart

Selected ES 12-09 with default to 5 days of data. Chart loaded ok.

On the same chart, selected "data series" and changed the days of data to 50. Chart loaded data OK in about 35 seconds.

Changed the days of data to 100. After a few minutes application pop-up error box appears informing that the application must shut down. Selecting OK doesn't allow the application to shut down, another pop-up instantly appears, with the same notification. Application terminated via task manager.

--------------------------
2009-12-04 11:35:19:000 (Mirus Futures) Cbi.ConnectionStatusEventArgs.Process: Connected Connected
2009-12-04 11:35:19:015 (Mirus Futures) defaultTimeToClose=12/4/2009 12:00:00 AM flattenAll=False
2009-12-04 11:40:24:890 (Mirus Futures) Data.Bars.GetBarsNow: instrument='ES 12-09' from='2009-11-29' to='2009-12-03' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2009-12-04 11:40:32:703 (Mirus Futures) Data.Bars.GetBarsNow: instrument='ES 12-09' from='2009-12-04' to='2009-12-04' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2009-12-04 11:40:38:500 UI (re)load complete, took 14.1 secs, 636.9 secs after app start
2009-12-04 11:40:55:562 (Mirus Futures) Data.Bars.GetBarsNow: instrument='ES 12-09' from='2009-10-15' to='2009-12-03' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2009-12-04 11:41:51:453 (Mirus Futures) Data.Bars.GetBarsNow: instrument='ES 09-09' from='2009-08-22' to='2009-09-09' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2009-12-04 11:42:01:125 (Mirus Futures) Data.Bars.GetBarsNow: instrument='ES 12-09' from='2009-09-06' to='2009-12-03' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2009-12-04 11:45:08:515 in OnUnhandledThreadException
2009-12-04 11:45:08:625 ********* exception trapped *********
2009-12-04 11:45:08:625 Parameter is not valid.
2009-12-04 11:45:08:625 at System.Drawing.Pen.get_Width()
at NinjaTrader.Gui.Chart.CandleStyle.GetBarPaintWidth (Int32 barWidth)
at NinjaTrader.Gui.Chart.ChartControl.GetBarPaintWidt h(Bars bars)
at NinjaTrader.Gui.Chart.ChartControl.InitBarsPainted ()
at NinjaTrader.Gui.Chart.ChartControl.OnPaint(Object sender, PaintEventArgs e)
at System.Windows.Forms.Control.OnPaint(PaintEventArg s e)
at System.Windows.Forms.Control.PaintWithErrorHandlin g(PaintEventArgs e, Int16 layer, Boolean disposeEventArgs)
at System.Windows.Forms.Control.WmPaint(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ScrollableControl.WndProc(Mes sage& m)
at System.Windows.Forms.Control.ControlNativeWindow.O nMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.W ndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
--------------------------

Cheers,
Paul.

[NinjaTrader_Austin]

Paul, I was unable to reproduce this error with the steps you listed. Are you running any indicators on your chart?

Just to clarify, you're using a chart of ES 1 Minute (as indicated in the picture of your most recent post) correct?

Either way, loading up 100 days of tick data (especially on ES!) is extremely CPU and RAM (bandwidth as well) intensive, and we don't recommend using more than 5-10 days of tick data at a time.

Anyways, while sitting and thinking out the rest of this post, I decided to load 100 days of 1000 tick ES bars. It took a while (4-5 minutes) but it loaded completely and correctly.

A quick recommendation: buy more RAM for your computer. Mine has 4GB, which appears to be enough.

[bayofislands]

Hi Austin,

As you can see from the attachments supplied, and the comments in my previous reply. This is a fresh install, the procedures described are those effected from this new install, ie no indicators or other setting changes.
There should be no increased bandwidth or effort required to load the 50 to 100 days than there was the 0 to 50 days. There certainly shouldn't be the application failure, which is why it is brought to your attention? This is an HP xw4400 with 4GB (only 3.6GB available under XP tho) has proved quite adequate for NinjaTrader 6.5 and 7.0.

Regards,
Paul.

[NinjaTrader_JoshP]

Paul,

When you installed NT7, did you migrate 6.5?

[bayofislands]

Hi Josh,

It has been a complete re-install of version 7.0.0.5

Version 6.5 and 7.0 completely removed, with ancillary local directories, before the new installation.

Look this is something I can live with to continue testing other parts of your application, I don't load 100+ days of tick/minute data in normal use.

I am nonetheless greatly disturbed that the application should fail in the manner it does when such a historical request is made, especially as the failure occurs occasionally over modest look back periods of a few days.

It isn't a lack of memory issue as strangely intimated by Austin. Maximum memory usage has been about 150MB so far, leaving over 2GB free!

Cheers,
Paul.

[NinjaTrader_JoshP]

Paul,

I fully understand, but we need a reproducible scenario before we can try to isolate what could be the cause. As such, any information you can provide as to how you reproduce will be useful since none of our test machines or other peoples machines show this issue.

Please provide exact steps as to how you have installed and reproduced. To be doubly sure, please ensure you have removed the Documents\NinjaTrader 7 directory on uninstall prior to reinstall.

[bayofislands]

Hi Josh,

I have uninstalled the application completely, both 7.0 and 6.5 before re-installing 7.0.0.5 and this is the third time I have explained this.

Please see reply #6 it outlines the procedure taken.

I did not attempt to hand check the registry to see if there was some stale remnants left after the uninstall, but I did check the "program files" and the local "my documents" directory.

THIS WAS A COMPLETE UNINSTALL AND CLEAN INSTALL

Cheers,
Paul.