Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Importing Historical Data - Associating the Data with a Specific Contract

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Importing Historical Data - Associating the Data with a Specific Contract

    Hello,

    I have purchased about 20 years worth of historical tick data for NQ, and imported it into Ninjatrader. However, I see that the data is all lumped into its own "Instrument", but this new instrumnent is not available in the strategy backtester. How can I either have Ninjatrader associate the imported data with existing instruments, or be able to select the new custom instrument in the strategy analyzer.

    Click image for larger version  Name:	2023-04-09_11h57_05.png Views:	0 Size:	49.5 KB ID:	1245292


    Thanks!

    #2
    Well, you have two choices here - you can rename the file and import it into a "real" instrument that already exists or you can add the details for the NQ_TICK_NT instrument to the instrument list. I would recommend you do the latter, so you don't have a hodge-podge instrument in which some data is from CQG and some from the file or something like that - that has the potential to waste a lot of your time trying to figure out why it "breaks" on a certain date. What you probably should do is to add that NQ_TICK_NT to the instrument list if it isn't already and fill in all of the details, copying them from the NQ future definition such as the tick size, point value, etc. and then you can pick the instrument NQ_TICK_NT in strategy analyzer.
    Bruce DeVault
    QuantKey Trading Vendor Services
    NinjaTrader Ecosystem Vendor - QuantKey

    Comment


      #3
      Thanks Bruce!

      It looks like it already existed in the instrument list, as Type = Stock. So I manually added a new one with the same name, with the same details as NQ. However, when I select the one I manually added from the instrumnet list in strategy analyzer, it gives me the message "Please select an instrument to run a backtest on"

      Not sure if there's another step I'm missing. Do I need to delete the instrument that was auto generated as type = Stock?

      Click image for larger version

Name:	2023-04-09_13h12_17.png
Views:	128
Size:	34.9 KB
ID:	1245302

      Comment


        #4
        Delete the stock one. That is what happened when you ran the import without having it in the instrument list.
        Bruce DeVault
        QuantKey Trading Vendor Services
        NinjaTrader Ecosystem Vendor - QuantKey

        Comment


          #5
          Ok, deleted that one. Now I'm getting a new error:

          Click image for larger version  Name:	image.png Views:	0 Size:	15.0 KB ID:	1245327

          So far I only added a single contract ​month to this instrument, the current contract. I get this error if I try to run a backtest or just open a chart for this instrument.
          Last edited by beejinator; 04-09-2023, 02:42 PM.

          Comment


            #6
            The one month you added - is it 06-23 and you set a rollover date that is in the FUTURE?
            Bruce DeVault
            QuantKey Trading Vendor Services
            NinjaTrader Ecosystem Vendor - QuantKey

            Comment


              #7
              No, I used the same rollover date as the 6-23 NQ - 3/13/23

              Comment


                #8
                Hello beejinator,

                Thanks for your notes.

                It seems you have created the instrument as an equities instrument in the Instruments window and not a future.

                You would instead need to create a futures instrument in the Instruments window to add your instrument. Also, the historical data text file that you are importing has to have the contract month in the file name.

                And, you would need to add the contract months to the instrument once it has been made as a futures instrument.

                Working with Instruments: https://ninjatrader.com/support/help...e_database.htm

                Please let me know if I may assist further.

                Brandon H.NinjaTrader Customer Service

                Comment


                  #9
                  I did create it as a futures instrument - you can see in my earlier screenshot (the instrument underneath the green circled one) it says Futures in the Type column. Apologies, I'm not quite following your instructions.

                  I think it would be easiest if we maybe start over, and you can create a step by step list of all things that need to be done on my end to get this imported correctly? Please create the list of steps with the assumption that my data is all in a single file (20 years / 800 million rows) and it's not realistic to manually work with this file as it's too large to open in programs like excel.

                  Thanks so much for all your help

                  Comment


                    #10
                    In situations like this, I create an artificial futures instrument, then create a single rollover set to happen in the future (no past months), and then import all of the data into that one month.
                    Bruce DeVault
                    QuantKey Trading Vendor Services
                    NinjaTrader Ecosystem Vendor - QuantKey

                    Comment


                      #11
                      Ok thanks! I created a new instrument called NQ_Historical, which will also be the name for the text file I will re-import. The instrument has been duplicated from the system NQ instrument, and I've added a single future rollover month. Is there anything else that needs to be done for this to import properly and be a usable instrument for backtests?

                      New Instrument:
                      Click image for larger version

Name:	2023-04-11_16h23_26.png
Views:	109
Size:	113.9 KB
ID:	1245706

                      Single Rollover Month Dated in the Future:
                      Click image for larger version

Name:	2023-04-11_16h23_14.png
Views:	104
Size:	29.5 KB
ID:	1245707​​

                      Comment


                        #12
                        Hello beejinator,

                        Thanks for your notes.

                        If you are importing historical data for a futures or forex instrument, the instrument MUST exist in the database. If it does not, you must add it first via the Instruments window before importing the historical data. This could be seen in the Importing Tips help guide below.

                        Importing Tips: https://ninjatrader.com/support/help...calData​

                        When you import historical data for a futures instrument, the text file needs to include the expiry month. Please see the link below under 'Understanding import file and data formats'.
                        https://ninjatrader.com/support/help.../importing.htm

                        For example:
                        MSFT.Last.txt for Microsoft stock last price data
                        ES 12-09.Bid.txt for the S&P E-mini December contract bid price data
                        EURUSD.Ask.txt for the Euro/U.S. dollar currency pair ask price data​

                        Please let me know if I may assist further.
                        Brandon H.NinjaTrader Customer Service

                        Comment


                          #13
                          Hello NinjaTrader_BrandonH and QuantKey_Bruce - I have followed both of your advice, but unfortunately still having issues. I took the following steps:
                          1. Created a new futures instrument, using the same settings as the existing NQ instrument, named "NQ_HISTORICAL"
                          2. Within this new futures instrumnet, I created one single future dated rollover for December 2050, "12-50"
                          3. Renamed my import file (~20years of tick data for the NQ) to "NQ_HISTORICAL 12-50.Last.Txt"
                          When I run the import again, instead of importing the data as it did last time, I receive this error message:
                          Click image for larger version

Name:	2023-04-14_10h18_07.png
Views:	109
Size:	45.0 KB
ID:	1246232

                          Comment


                            #14
                            Hello beejinator,

                            Thanks for your note.

                            It is likely that the data you are trying to import is not in a correct format. You would need to make sure that the data you are importing in the file is in one of the formats listed in the help guide page below.

                            Understanding import file and data formats: https://ninjatrader.com/support/help...AndDataFormats
                            Formatting data and formatting the file name: https://ninjatrader.com/support/help...romTickDataLLC

                            Further, I see you noted that you created a single rollover date for December 2050 which will not work. You should just add the current year that some of the data exists in. For example, if the data goes up to this month add a contract for this month.

                            The future instrument expects to have multiple contracts of data and to use the merge policy with offsets. If your data is already merged going back many years and isn't something to be traded on/fake instrument, it may be easier to just use a stock type instrument instead of a futures instrument.

                            Let me know if I may assist further.
                            Brandon H.NinjaTrader Customer Service

                            Comment


                              #15
                              I was able to get it to work by removing ".last.txt" from the file name. In this case, it seems that the instructions sent were misleading. Might want to update the documentation to include correct file format for tick data, as well as tick-millisecond data. In my case, using tick data, adding the ".Last.txt" was breaking the import.

                              Working as expected now, all data imported and usable for backtests.

                              Thanks for the help!

                              Comment

                              Latest Posts

                              Collapse

                              Topics Statistics Last Post
                              Started by WHICKED, 06-27-2023, 12:07 PM
                              66 responses
                              768 views
                              1 like
                              Last Post NinjaTrader_ChelseaB  
                              Started by tkaboris, 06-11-2024, 08:06 AM
                              11 responses
                              32 views
                              0 likes
                              Last Post tkaboris  
                              Started by Matt Skinner, Yesterday, 12:23 AM
                              3 responses
                              28 views
                              0 likes
                              Last Post Matt Skinner  
                              Started by Dan_TUK, Today, 05:14 AM
                              0 responses
                              7 views
                              0 likes
                              Last Post Dan_TUK
                              by Dan_TUK
                               
                              Started by wzgy0920, 04-20-2024, 06:09 PM
                              7 responses
                              59 views
                              0 likes
                              Last Post Joia_
                              by Joia_
                               
                              Working...
                              X