Best practice for rapidly acquiring a good testing pool of tick replay data?

[carnitron]

So far, the only way I know of to get tick replay data I can use is to bring up a chart window with the ticker of interest during market hours, while I have the "Tools/Options/Market data/Enable market recording for playback" checkbox checked.

I have tried downloading data from TD Ameritrade (my backend) and NinjaTrader via the "Tools/Options/Market data/General/Preferred connections - historical/Stock" and "Tools/Options/Market data/General/Preferred connections - real time/Stock" settings, but I have never been able to successfully get downloads at all, no matter which ticker or when I try it.

Presumably this is b/c tick replay data isn't available from TDA or NinjaTrader?

Would my best option be to pay for a month of Kinectick and just go wild downloading data during that month?

Are there other options?

Cheers.

[NinjaTrader_Manfred]

Hello carnitron,

Thank you very much for your post.

TD Ameritrade, unfortunately, does not provide historical tick data through the API.

As minute data is the smallest granularity provided, this would not suffice for market replay. As such, leaving NinjaTrader connected and recording this live data as market replay data would be the only option outside of connecting to a provider that supports historical tick data (e.g. Kinetick).

Please don't hesitate to let me know, should you have any further questions!

[carnitron]

Ah ok, that is what I suspected. Thx for confirming. Time to try Kinectick!

[carnitron]

So I now have the basic $75 Kinetick plan and I am currently logged into my Kinetick account via NT8. I still can't seem to download Market Replay data.

This is what I get:



These are what seem like the relevant settings:



On the upside, I appear to be able to download Historical Data:



I'm under the impression that Market Replay data is more accurate and complete than Historical Data, so it seems like I should prefer that.

What am I missing here?

[NinjaTrader_Adrian]

Hello carnitron,
Thank you for your post.

Please note, if you have the Kinetick basic subscription that only includes delayed data which is why the market replay is not working. The historical data is working because you do not need to have an active data connection to download historical data from within the platform. For Kinetick all futures are 10 minutes delayed data if you do not have real-time, Equities are 15 minutes, and the CBOE Indexes (SPX.XO, VIX.XO, etc...) are 20 minutes. Please reach out to [email protected] directly in order to enable real time data on your Kinetick account or for any further questions related to this topic.

Please let us know if we may provide any further assistance.

[carnitron]

Ok, I can definitely reach out to Kinetick support to enable real time data if I need to do that. But do I need to do it? Yes, the basic market data is delayed by however many minutes, but are you saying that b/c of that Market Replay data doesn't work at all?

Why would that matter if I am trying to download Market Replay data from yesterday, for instance?

Or are you saying that the delay breaks market data entirely, b/c it's no longer "real" data at that point?

[NinjaTrader_Adrian]

Hello carnitron,
Thank you for your reply.

To clarify from my previous response, Market Replay data can be downloaded from NinjaTrader servers (free), though we offer a limited amount of market replay data. Only the most common futures and forex instruments are currently available, and equities are unavailable for download.

For equities, we recommend running the Playback Connection off of Historical Tick data (Historical playback mode) from Kinetick, instead of Market Replay.

I have provided a publicly available link below to our YouTube video that provides a brief overview of downloading Market Replay data, connecting to the Playback Connection, and replaying data:

• https://www.youtube.com/watch?v=CktTvESKo4w

Please let us know if we may be of any further assistance.

[carnitron]

1. Ok, understood that NT does not provide equity historical data. My experience certainly confirms this.

2. "For equities, we recommend running the Playback Connection off of Historical Tick data (Historical playback mode) from Kinetick, instead of Market Replay."

Just so I understand, why is this? What are the pros and cons of one vs the other?

3. The video link you provided sounds awesome though it currently marked as private, so I cannot view it. Can you make it public?

Cheers.

[NinjaTrader_Adrian]

Hello carnitron,
Thank you for your response.

Historical data is less accurate than market replay data, less granular, and no level two market depth information (ex. Chart Trader - No movement in Ask/Bid). The plus side to using historical data is that you can download multiple days at a time and typically historical data will go back about a year for tick data, minute data back through 2006 and daily data will go back through 2009, though data prior to 2016 will just be for regular trading hours (RTH).

Market Replay data would be more accurate and granular than historical data plus level two market depth information would be visible. The drawback to using market replay is that you can only download one day at a time and you can only go back 90 days for downloading market replay data.

Here is the video linked below:

• https://www.youtube.com/watch?v=CktTvESKo4w

Please let us know if we may provide any further assistance.

[carnitron]

So I'm still in a situation where I have never been able to download Market Replay data. Even with my new $75 base Kinetick subscription. I would like to get this working so I don't have to record everything manually. I also think I may need to go this route over Historical Data, b/c of the way parts of my algo work.

What do I need to make this work? Do I need real time data from Kinetick? If so, what is the minimum complete set of options I need to purchase from Kinetick?

Moreover, at this time point in the video: https://youtu.be/CktTvESKo4w?t=60

It says "Playback data is provided free of charge to all NinjaTrader users, including those using simulator license keys."

So what are the setting I need to use to get this data?

I have only ever gotten the dialog with the error:

"There is no market replay data available for the instrument/date."

Also, to follow up from earlier in the convo here: why does NT8 recommend Historical Tick Data over Market Replay data?

[NinjaTrader_Adrian]

Hello carnitron,
Thank you for your post.

Yes, the playback data is always free of charge and only the most common futures and forex instruments are currently available and equities are unavailable for download with market replay. You are receiving that error message because you are trying to view equities and that is currently not supported at this time with market replay. We recommend using historical data over market replay for equities because that is the compatible version that is needed in order to view the equities data in playback.

In order to resolve your issue, you simply need to download your equites data using the historical data option. Once downloaded this will playback using our playback controller, in addition if you wish to view futures data then you may use market replay for that since only futures and some forex instruments are available with this option. When using the playback controller please ensure you have the 'Historical' or 'Market Replay' option selected to correspond with whatever data you are trying to load in the playback controller.

See this help guide for details about downloading Historical and Market Replay data: https://ninjatrader.com/support/help...8/download.htm

And, see this help guide page about using the Playback connection: https://ninjatrader.com/support/help...ePlaybackWorks

Please let us know if we may provide any further assistance.

[carnitron]

Right, you did mention earlier that market replay data for equities were not available from NT, which I forget when replying here.

"We recommend using historical data over market replay for equities because that is the compatible version that is needed in order to view the equities data in playback."

So I want to be very clear on what you mean here, b/c I am able to use Market Replay Data for equities that I record myself, meaning that NT8 is able to use Market Replay Data for equities in at least some contexts. Are you actually saying that downloading Market Replay Data for equities is not compatible and will never work?

The guy in the video is momentarily searching for AAPL for Market Replay Data, and the interface itself certainly enables you to type in equities for Market Replay Data download, so I'm sort of skeptical this is the final answer.


But if we run with that model of how this works, would it be fair to say:

1. I actually got my $75 Kinetic subscription to get Historical Data. There are no other options I can buy from them that will enable me to download Market Replay Data for equities.

2. The only way I can use Market Replay Data for equities is to record it myself.

[NinjaTrader_Adrian]

Hello carnitron,
Thank you for your reply.

Yes, downloading equities will not be possible using market replay data since stock replay data is not available from NinjaTrader replay servers. Market replay doesn't come from the provider - it comes directly from NinjaTrader servers. When using the Download Replay Data tool, only Futures and the most common Forex pairs would be available for download. The technician in the video was simply showing you that you can search for instruments within the instrument list using the historical data window.

To resolve your issue, you would either need to record market replay data yourself or use the historical data option after loading data from your provider.

Please let us know if we may provide any further assistance.

[carnitron]

Ok that clarifies things. Thank you for sticking with me here as we got to the bottom of this. Cheers.