Historical Data Manager

[Harry]

Tried to build daily bars from intraday data for EURUSD. To achieve this I downloaded 2 years of minute data from NinjaTrader/Gain servers.

Then I tried to convert this to daily data, which resulted in an error message. Checking the logs, I find the following error message for each day that I tried to build:

06.02.2010 01:42:45|3|4|Unable to import file 'C:\Dokumente und Einstellungen\Harry\Eigene Dateien\NinjaTrader 7\db\minute\$EURUSD\20090330.Last.ntd'. Instrument is not supported by repository.

Any ideas, why it did not work?

[NinjaTrader_Ray]

The file you are trying to import is not a valid import file/format.

- Export your data via the Historical Data Manager
- Then import back this exported data

That should do the trick.

[Harry]

Thank you for the hint. This worked indeed.

Is it correct that the daily data is generated from the intraday data by using the session specified under instrument settings?

The file you are trying to import is not a valid import file/format.

- Export your data via the Historical Data Manager
- Then import back this exported data

That should do the trick.

[NinjaTrader_Dierk]

>> Is it correct that the daily data is generated from the intraday data by using the session specified under instrument settings
correct

[Harry]

I am impressed with this feature

That's really great.

I just converted the intraday FOREX data from the Gain/NinjaTrader to daily data and stored in the data base. I also observed that the daily close was identical with the session close at 10:00 PM GMT.

With daily data going back to mid-2006, I can now run my fibonacci cluster indicators on an intraday chart with 45 days lookback period and using daily data back over 1000 days to calculate retracements and extensions from major turning points... Waiting for the next release to cope with the deadlock that sometimes occurs when loading daily data.

[Harry]

Database editing allows to correct Gain/NinjaTrader spikes

Just checked the FOREX minute data from Gain/NinjaTrader servers back to June 2006 for the six major currency pairs. I found the following bad ticks:

USDJPY 4 Feb 2008 22:01 GMT High 210.55, Low 96.87
USDCHF 10 Sep 2007 8:40 GMT Low 0.5924

Maybe you want to leave these on your servers, so that other users can also try to remove them manually?

With NT7 no problem to edit the database and to remove the offending spikes. If I had not removed those spikes, a Fibonacci indicator with a lookback period of 3 years would only display nonsense, i.e. calculate the retracement from the spike, except when using a bad tick filter within the indicator.

[Harry]

Here is a small observation:

When opening a range chart for EURUSD today, connected to Gain, historical data was loaded for Sunday evening through today as specified by the FOREX session template. However, Friday data was only shown partly on the chart. So the backfill did not complete Friday's session.

Reloading all historical chart data filled the gap, but I do not understand, why this gap was left open in the first time (see chart below).

[NinjaTrader_JoshP]

Harry,

Unfortunately I was unable to reproduce. Are you able to? Perhaps you could provide steps we could follow if so. Thanks.

[Harry]

Josh,

I have another case here: NT 7.0.0.9 connected to TWS 900.6:

The screenshot below shows two charts opened simultaneously. Both charts use the same session template <instrument settings>. Both charts show ZB 03-10. I was just comparing two versions of pivot indicators, that produced different lines. Then I discovered that the reason for the difference was simply missing data on the left chart. All 6 pivot indicators were running in CalcFromIntradayData mode, so this problem is not linked to the daily data bug.

Refreshing via F5 would not change anything. Reloading data did solve the problem.

How can this happen? I need to rely on the data and can not afford that NinjaTrader displays the data in a different fashion every time a chart is loaded.

False Data -> False Trading Signals -> Losing Trades!

[NinjaTrader_JoshP]

Harry,

Please provide reproducible steps in getting up a chart. Thank you. Chart is only loaded if you hit these rules: http://www.ninjatrader-support.com/H...HistoricalData

[Harry]

How shall I reproduce? I am opening two (near identical) charts, one after the other, and they are showing different price information, as you can see. The only way to reproduce this is to open hundreds of charts with NinajaTrader until it occurs again. And then again, you have no clue.

Harry,

Please provide reproducible steps in getting up a chart. Thank you. Chart is only loaded if you hit these rules: http://www.ninjatrader-support.com/H...HistoricalData

[NinjaTrader_JoshP]

Harry,

When you reproduce, save the workspace and send us the workspace and your database for analysis. There is little more I can hint you with as I would not know how you achieved the results at this point in time. Thank you.

[Harry]

Will send you workspace, if problem occurs again. Is not easily reproducible.

Harry,

When you reproduce, save the workspace and send us the workspace and your database for analysis. There is little more I can hint you with as I would not know how you achieved the results at this point in time. Thank you.

[Harry]

Data gaps again and again.

This occured again several times today. See chart below. I forgot to save the workspace, but it was

DOM and 2 default charts 60 min, 65 days, both without any other indicators (except my candle colours).

The $AUDUSD chart displays 2 gaps (screenshot 1). I checked the IB chart, data was complete. I closed down NT and then restarted without reloading. One of the gaps closed (not connected) the other remained (screenshot 2). The second gap was closed when I toggled from 60 min to 30 min.

What does this mean?

[Harry]

Now the story continues. After connecting to IB the GAP reoccurs. I am now connected and can toggle between a 30 min and a 60 min chart.

30 min chart : complete data! (screenshot 1)
60 min chart : 10 hours of data missing! (screenshot 2)

I have toggled back and forth about 20 times, always like this. Besides, now I have just the control center and one default chart open. Nothing else! Below are my screenshots, you will notice the entire workspace is just one default chart.

Now comes the catch: If I open the historical data manager, there is no data prior to 9:51 for Feb 16 (screenshot 3) This is true in both cases, when the 30 min chart is open and when the 60 min chart is open.

So, if there is no data that can be edited with the data manager, what data is displayed on the 30 min chart?

Lots of interesting questions. I will send you via mail the part of the data base of NT7 that contains all the AUDUSD data, so you can check it.