SQL Practical limitations, record full level2 depth, Excel RTD?

[Drmayer]

1. What are the practical limitations of NT8 data recording?
-how many real time symbols?
-does it record full bid/ask depth or just closest bid/ask. And is each change in bid/ask recorded or just at last trade?
2. Can data be accessed & streamed to Excel via RTD or other means?
3. Does a user need a special developer license to create new columns in DOM?
Thanks,
Drew

[NinjaTrader_Brett]

Hello,

1. We don't have any hard data on this at this time with NinjaTrader 8. We record to a proprietary format (Not SQL) and the limitation is purely disk read/write speed in most cases. We have 3 data streams, last, bid, and ask which can be recorded. This is primarily the same as NT7.
2. Not by default but you could create your own developed addon to do so.
3. No everything is available there is no 'locked' features for development. We are aiming to become a complete open development platform where basically anything is possible.

[Drmayer]

Thanks Brett. Just a few follow up questions to see if I understand correctly:
4. Only Level 1 (including bid/ask size?) is recorded to the database?
5. Even though Level 2 flows through to the DOM it is not saved to the database?
6. Is Level 2 written to a temp file or does it only reside in RAM & then discarded?
7. Can Level 2 be accessed in other areas of NT8? i.e. Indicators or charts?
8. Are you aware of anyone that's figured out how to record Level 2 data & play it back into NT through to the DOM, charts, or indicators?
9. Are you aware of anyone that has created an Excel interface for NT?
Thanks again,
Drew

[NinjaTrader_Ray]

Thanks Brett. Just a few follow up questions to see if I understand correctly:
4. Only Level 1 (including bid/ask size?) is recorded to the database?
5. Even though Level 2 flows through to the DOM it is not saved to the database?
6. Is Level 2 written to a temp file or does it only reside in RAM & then discarded?
7. Can Level 2 be accessed in other areas of NT8? i.e. Indicators or charts?
8. Are you aware of anyone that's figured out how to record Level 2 data & play it back into NT through to the DOM, charts, or indicators?
9. Are you aware of anyone that has created an Excel interface for NT?
Thanks again,
Drew

4. Yes but it is recorded to a compressed format to a file
5. Correct unless you enabled recording for Market Replay purposes, but if recorded it is not accessible
6. Discarded after use
7. Yes, there is OnMarketDepth() handler
8. That is what our "Playback" using recorded Market Replay does
9. I believe I have seen people discuss it in various forums but don't have direct knowledge off the top of my head

[Drmayer]

Thanks Ray.
I think I understand most of your answers, but I'm confused about your answer to #5...
Level 2 data can be recorded to NT8 database if "Record for Market Replay" is selected?
But if recorded, Level 2 data it is not accessible? (Why record it if it can't be accessed?)
Is the recorded Level 2 data not accessible or is the real-time Level 2 data stream not accessible while it is being recorded?
Thanks again Ray,
Drew

[NinjaTrader_Matthew]

Market Replay is a proprietary system used to "Playback" market data events just like it was live. This is useful for education purposes as well as backtesting strategies to see how they would have performed historically in a simulated "real time" environment.

You cannot access this recorded market replay L1 or L2 data for other uses.

You can access the real-time L2 stream while it's being recorded for market replay.

[James108]

[QUOTE=NinjaTrader_Matthew;411438]Market Replay is a proprietary system used to "Playback" market data events just like it was live.

Mathew

Now that NT8 supports using Historical data for replay what are the reasons I might still use market replay if I have the required historical data.

Basically what is the different in functionality?

thanks

[NinjaTrader_Matthew]

"Historical" is less accurate, and does not provide Level 2 data.

Historical bid/ask volume is synthetic and not reflective of true market conditions.

[Drmayer]

Thanks Matthew,
I appreciate your, Ray's, & Brett's answers.

Mostly I plan to use NT8 for market depth studies, but the fact that NT7 users are using bid/ask depth recorders like GomRecorder, BaBAR, etc. raised questions. I think I'm getting the gist of it, but don't want to draw wrong conclusions so these should be my last questions about it:

10. Were workarounds like GomRecorder & BaBAR necessary due to NT7's database limit of 1 second granularity?
11. Has the change to millisecond granularity in NT8's database corrected that?
12. Other than that improvement, everything else I read about NT7 data handling should be the same as NT8?

13. Bid/ask depth data will not populate a historical chart, or indicators on a historical chart? It can only stream through & be processed within indicators in real time or by Market Playback?

14. Can the bid/ask depth data, either streaming fresh in real-time or by Market Replay, be used across the NT8 platform in much the same way as price data? In other words, other than in the DOM, can it be used to build Charts? Indicators? Strategies? or in the Market Analyzer?
Or more precisely... within the NT8 platform, where can bid/ask depth data be processed (perform calculations upon it) & where can it be visualized (displayed graphically)?

Thanks for your patience in explaining it to me,
Drew

[NinjaTrader_Brett]

I have a feeling your still not 100% on the right track and that a 5 minute phone conversation could clear it up. If you'd like to do that please email me at support [at] ninjatrader [dot] com ATTN: Brett and reference this forum post and include your phone number. I'll work to get you headed in the right direction.

To answer your questions directly however:

10. One reason but not the primary reason, it was to record the OnMarketData events so they could be played back to rebuild an indicator/strategy like it seen the data live. (Not off purely historical tick data)
11. Not in all cases. Since it A) Wasn't the primary issue and B) not all providers supply data in that format.
12. For the most part except I want to make sure your aware of the new Tick Replay feature in NT8. Which I also want to make sure you don't confuse with market replay in NT7.
13. Not sure what you mean can you please clarify.
14. You'll probably want to use the new NT8 Tick Replay feature not market replay. Again I believe a quick phone conversation is needed to clear up all the info.

[James108]

Brett
I hope there is a comprehensive description as to how all theses various mode work compared the NT7, something like a white paper would be good. I am still confused about the various capabilities of the different modes of playback.

James108

[NinjaTrader_Brett]

Hello,

Yes we do have plans to beef up the NinjaTrader 8 documentation on this prior to release of open beta.

[Drmayer]

Just got off the phone with Brett & am looking forward to buying an NT8 lifetime license. I became futures licensed in 1984, used CQG, then was one of the Cruz brothers 1st customers with SystemWriter, then Tradestation, have used other platforms through the years, have investigated countless others & I gotta say you NT guys are good, very good... including the user base here & at BigMike's.
Thanks guys, you've made a convert!
Drew