Not able to load Historical Bars in Playback

[Bluebeep]

I'm installing 4 machines with the same workspace and configuration for a test and each machine has the same replay data loading the same days. However for one of the machines I can't seem to get the historical bars to load.

I have checked the bars to load, changed from Days to Bars and I have it 'kind of working'. I seem to be able to pull up data upto midnight of the same day if I switch to 'bars' instead of 'days' but as soon as I change the time I have to change the bar count again.

I've tried repairing the database, reinstalling NinjaTrader, redownloading all the replay data from our server, testing on other machines... I'm just stumped.

Not sure if I'm even making sense so I made a video. If a picture is worth 1,000 words, hopefully the video will save us a month of talks

I broke my NinjaTrader! Thanks for any input guys!

https://youtu.be/B3LAHy04HEY

[NinjaTrader_ChelseaB]

Hello Bluebeep,

Historical data is downloaded from a connected provider such as the NinjaTrader (brokerage) connection?

May I confirm you are downloading the historical data before connecting to Playback to playback Market Replay data?

May I confirm you are also updated to the latest release 8.1.1.3? (Help -> About)

https://account.ninjatrader.com/download

[Bluebeep]

Hi NinjaTrader_ChelseaB,

At the end of the day one of the guys downloads Replay Data from Continuum. This is copied to the server and then copied over to 3 other machines the next morning that run a strategy against certain dates.

For this particular machine, we did nothing different. Installed NinjaTrader, copied over the Replay files from the server and after your note I verified that the Replay Data exists in the users \Documents\NinjaTrader 8\db\replay.

The version of NinjaTrader were using is 8.0.27.1. I did try to work with the latest greatest but it seems like I need to get some sort of account on your website and were in the middle of testing and we have lifetime licenses so we all decided to stick with 8.0.27.1 for now until this round of our tests are done. So each machine has 8.0.27.1 installed for testing including this problem machine.

[QuantKey_Bruce]

Just to be clear, Replay Data comes from NinjaTrader's servers regardless of whether you are using Continuum as a data feed. You need the historical data (minute, day, or tick) as well not just the replay data to load a chart with historical bars before you start the replay.

[Bluebeep]

Hi QuantKey_Bruce,

Ok thanks for the clarification. We are running 5-minute charts on our strategies. For quite a while now all I have done is copy over NRD files that we have stored on the server and place it into the db/replay folder and I'm able to load up bars just fine. When I check the 'minute' folder I see small metadata files that seem to have been generated as we load up Market Replay. That's stuff I never put in there.

So just as with all computers configured right now (that are working fine), the problem computer had the same method of loading data.. Copy over dated folders such as MES 06-23 which have a bunch of NRD files in them.

[QuantKey_Bruce]

It was my understanding that if you have replay data only, and no historical data and are offline, then when you connect to the playback connector, you will not have any historical bars on your chart or any context - but you can start the replay and it will start building from there. The first bar will be the bar of the start of the replay. If you opened the chart while online with a connection it can get historical data from e.g. Continuum etc, it would get that data BEFORE the playback connection is switched to just by opening the chart but that's a separate process from replay.

[Bluebeep]

Hi QuantKey_Bruce,

That's interesting. Maybe what I should do is copy over the minute folders from another working machine and see if that changes things. I'll report back as soon as I have had a chance to do this

[NinjaTrader_ChelseaB]

Hello Bluebeep,

Market Replay data is not historical data. These are not the same. Market Replay data cannot be downloaded from a connected provider. This is downloaded directly from the NinjaTrader Market Replay servers.

Market Replay data is a recording that can be played back. Bars will start building from the moment you click play. This does not backfill the chart.

Historical data is what backfills a chart. This is what is has to be downloaded before connecting to Playback.

What is the bar type on the chart?
What is the End date set to in the Data Series window?
How many days to load is set?

May I have you provide a screenshot of the Tools -> Historical Data window with the Historical section -> Instrument -> market data type -> interval data type -> date expanded so we may confirm the historical data has been downloaded from the provider?

[Bluebeep]

Hi NinjaTrader_ChelseaB,

Thanks for the update. I think I'm a little more confused now tho.

So if Market Replay is something that is generated from the moment I hit play, and Historical Data is something that is backfilled on the chart, how is it possible that I install a machine with only downloaded replay data and it backfills the charts? I built a new Virtual Machine (clean windows install), downloaded JUSt the NRD files, loaded up a chart, and I have 5 days worth of backfilled data. For the problem machine I do the same and there is no backfilling happening.

Here are differences between the two machines:
- Working Machine is a VM on an ESXi Server (Problem machine is a PC)
- Working machine has a Lifetime license installed (Problem machine is using the free version)

Does historical data not get downloaded if you're on a free version?

Where is this downloaded Historical Data stored? Maybe I can find a machine that has this historical data and is working correctly and copy it over to the machine that isn't.

The bar type is a Minute (5 Minute) - For both a working computer and the problem computer
The End Date is set to today - For both a working computer and the problem computer
Days to Load: 5 - For both a working computer and the problem computer

[NinjaTrader_ChelseaB]

Hello Bluebeep,

This is correct, downloading historical data automatically is linked to the license key in 8.0.27.1 and previous.

You will need to pre-download the historical data if you are using the Simulation or Free license keys.

Historical data is stored in Documents\NinjaTrader 8\db. However, I would recommend you only download from a connected provider.

Update to 8.1.1.3 and the data will automatically download.

[Bluebeep]

It's falling in place now! Thanks guys!

So for the time being (until we upgrade to 8.1.13, instead of just copying over just the replay data, I would also copy over the day, minute and tick folders?

If I have to pre-download historical data for the free license, how is that done? the 'Download' button is greyed out in my case.

[NinjaTrader_ChelseaB]

Hello Bluebeep,

We do not advise clients to manually move files in or out of the filesystem (with the rare exception of templates).

I would first recommend you just download it from a connected provider.

Otherwise I would direct you to export a backup file and then import this on the other computer.

Below I have provided a link to the Backup & Restore section of our help guide.

https://ninjatrader.com/support/helpGuides/nt8/backup__restore.htm

[QuantKey_Bruce]

These other computers could just sign up for a free account. They do not have to fund it. I don't really understand what's being gained by copying the files around - that seems like a bad idea to me too. Maybe an attempt to avoid paying the $4 or whatever for market data? If that is the case, you should pay it, because you are using it.

[NinjaTrader_ChelseaB]

Hello Bruce,

Historical data is free. Real-time data requires a subscription after the demo expires. But historical data is always available when connecting to the NinjaTrader Brokerage.

[QuantKey_Bruce]

Well, in that case, I completely agree - they should download it not copy the files around. Thank you NinjaTrader_ChelseaB for clarifying this.